[QUESTION] Recreating Figure 9 in the paper

[CPA872]

Your question
Dear Authors,

In Figure 9 of the paper, the Speedup is used as the x-axis. I am wondering what are the early-exit thresholds leading to the corresponding speedup? Can you shed light on this?

[pan-x-c]

In the experiment above, each line contains 21 points, and from left to right the threshold decreases by 0.05 each time from 1.0 to 0.0.

[CPA872]

Can you give a description of the evaluation environment? For example, what were the task prompts on the CNN/Dailymail dataset?

The provided checkpoint on ModelScope should be enough to create Figure 9. Is this correct?

Thank you for your time!

[pan-x-c]

We use an internally maintained version of HELM that could not be opened due to other work involved, but there are no changes to the prompts and the number of tokens compared to the official open-sourced version.

The provided checkpoint is enough to create Figure.9 with HELM.

[github-actions]

Marking as stale. No activity in 60 days.