Reproducing Table 7 (weight-only quantization results)

[SShock92]

Thank you for sharing the code of the awesome work!

I have difficulty reproducing weight-only quantization results in Table 7, especially W2A16 results.
How did you set the quantization grid?
I have 1) tried the standard Min-Max grid and 2) activated 'MSE' option in the weight quantizer, but the perplexity of quantized models is much larger than 100.

[sashkboos]

@SShock92

Thanks for your issue. Can you please put the complete list of arguments you are using (for example, the command that you run the code with) here?

[sashkboos]

Please check this.