[FT] Rerun evaluations with new metrics based on completions saved in details file

[JoelNiklaus]

Issue encountered

Rerunning an evaluation with a new metric requires rerunning the entire inference currently, which can be very costly.

Solution/Feature

It would be great, if we could specify a details file containing the predictions and use that to compute more metrics on.