One of the llama.cpp updates since last mikupad release seems to have broken top probabilities functionality

[Mithrillion]

Expected:
When mouse over a token, you should see the top 10 tokens at that generation step and their probabilities.

Observed behaviour:
When running with llama.cpp release b4281, using llama.cpp API option in mikupad[eb98d22], any generated text will only show the top tokens for the first token in the generated text for all tokens. Additionally, a JavaScript error is observed in browser console indicating e.completion_probabilities is missing. The error is also displayed under the "predict" button.

Possible workaround / clue for cause of issue:
I have only tested that llama.cpp release b4242 should still work. It must have stopped working since one of the later commits, but I do not have time to test for now.

[lmg-anon]

This seems to be a bug in llama.cpp, inspecting the API response it's possible to see that all received tokens have the probabilities of the first token, for example:

{
  "index": 0,
  "content": " was",
  "stop_type": "none",
  "stop": false,
  "id_slot": -1,
  "tokens_predicted": 5,
  "tokens_evaluated": 68,
  "completion_probabilities": [
    {
      "content": "K",
      "probs": [
        { "tok_str": "H", "prob": 0.42449134588241577 },
        { "tok_str": "In", "prob": 0.27832186222076416 },
        { "tok_str": "The", "prob": 0.11458157747983932 },
        { "tok_str": "K", "prob": 0.056616175919771194 },
        { "tok_str": "M", "prob": 0.046976491808891296 },
        { "tok_str": "A", "prob": 0.023433640599250793 },
        { "tok_str": "It", "prob": 0.02162409946322441 },
        { "tok_str": "As", "prob": 0.013689255341887474 },
        { "tok_str": "\"", "prob": 0.009486593306064606 },
        { "tok_str": "T", "prob": 0.004959092475473881 }
      ]
    }
  ]
},
{
  "index": 0,
  "content": " walking",
  "stop_type": "none",
  "stop": false,
  "id_slot": -1,
  "tokens_predicted": 6,
  "tokens_evaluated": 68,
  "completion_probabilities": [
    {
      "content": "K",
      "probs": [
        { "tok_str": "H", "prob": 0.42449134588241577 },
        { "tok_str": "In", "prob": 0.27832186222076416 },
        { "tok_str": "The", "prob": 0.11458157747983932 },
        { "tok_str": "K", "prob": 0.056616175919771194 },
        { "tok_str": "M", "prob": 0.046976491808891296 },
        { "tok_str": "A", "prob": 0.023433640599250793 },
        { "tok_str": "It", "prob": 0.02162409946322441 },
        { "tok_str": "As", "prob": 0.013689255341887474 },
        { "tok_str": "\"", "prob": 0.009486593306064606 },
        { "tok_str": "T", "prob": 0.004959092475473881 }
      ]
    }
  ]
}

[lmg-anon]

As for the e.completion_probabilities error, it should be fixed as of commit d99de77.