Here's some extensive speed testing I did

[BBC-Esq]

Here's some extensive testing that I did on some of the popular backends to run LLMs. Please see an analogous post on the llama_cpp repo for the actual scripts.

Here's the link to the explanations:
ggerganov/llama.cpp#6373

[Titus-von-Koeller]

Really cool @BBC-Esq, thanks for sharing!