Unhandled exception (JSON::ParseException) in AI Integration

[schniggie]

Describe the bug
Using the AI Integration results in an Unhandled exception (JSON::ParseException).

To Reproduce
Steps to reproduce the behavior:

1. noir -b . -T --ollama http://192.168.178.250:11434 --ollama-model llama3.2:latest

Details

noir -b .  -T --ollama http://192.168.178.250:11434 --ollama-model llama3.2:latest

░█▄─░█ ░█▀▀▀█ ▀█▀ ░█▀▀█
░█░█░█ ░█──░█ ░█─ ░█▄▄▀
░█──▀█ ░█▄▄▄█ ▄█▄ ░█─░█ {v0.19.0}

⚑ Detecting technologies to base directory.
▲ No technologies detected.
  ➔ If you know the technology, use the -t flag to specify it.
  ➔ Please check tech lists using the --list-techs flag.
⚑ Start AI-based analysis as the --ollama and --ollama-model flags have been used.
⚑ Initializing analyzers
⚑ Analysis Started
  ➔ Code Analyzer: 0 in use
  ➔ AI Analyzer: Ollama in use
Unhandled exception: Unexpected char 'I' at line 1, column 1 (JSON::ParseException)
  from noir in '??'
  from noir in '??'
  from noir in '??'
  from noir in '??'
  from noir in '??'
  from noir in '??'
  from noir in '??'
  from noir in '??'
  from noir in '__crystal_main'
  from noir in '??'
  from noir in 'main'
  from /usr/lib/libc.so.6 in '??'
  from /usr/lib/libc.so.6 in '__libc_start_main'
  from noir in '_start'
  from ???

Here also compiled with all debug symbols:

/home/xxx/noir/bin/noir -b .  -T --ollama http://192.168.178.250:11434 --ollama-model phi4:latest -u https://www.xxx.app

░█▄─░█ ░█▀▀▀█ ▀█▀ ░█▀▀█
░█░█░█ ░█──░█ ░█─ ░█▄▄▀
░█──▀█ ░█▄▄▄█ ▄█▄ ░█─░█ {v0.19.0}

⚑ Detecting technologies to base directory.
▲ No technologies detected.
  ➔ If you know the technology, use the -t flag to specify it.
  ➔ Please check tech lists using the --list-techs flag.
⚑ Start file-based analysis as the -u flag has been used.
⚑ Initializing analyzers
⚑ Analysis Started
  ➔ Code Analyzer: 0 in use
  ➔ AI Analyzer: Ollama in use
Unhandled exception: Unexpected char 'T' at line 1, column 1 (JSON::ParseException)
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/lexer.cr:331:5 in 'raise'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/lexer.cr:327:5 in 'unexpected_char'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/lexer.cr:326:11 in 'unexpected_char'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/lexer.cr:243:10 in 'consume_number'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/lexer.cr:63:7 in 'next_token'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/parser.cr:108:11 in 'next_token'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/parser.cr:9:5 in 'initialize'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/parser.cr:6:3 in 'new'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json.cr:137:5 in 'parse'
  from /home/cd/hacktools/noir/noir/src/analyzer/analyzers/llm_analyzers/ollama.cr:40:26 in 'analyze'
  from /home/xxx/noir/src/analyzer/analyzer.cr:18:3 in '->'
  from /home/xxx/noir/src/analyzer/analyzer.cr:75:38 in 'analysis_endpoints'
  from /home/xxx/noir/src/models/noir.cr:99:18 in 'analyze'
  from /home/xxx/noir/src/noir.cr:125:1 in '__crystal_main'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/crystal/main.cr:118:5 in 'main_user_code'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/crystal/main.cr:104:7 in 'main'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/crystal/system/unix/main.cr:9:3 in 'main'
  from /usr/lib/libc.so.6 in '??'
  from /usr/lib/libc.so.6 in '__libc_start_main'
  from /home/xxx/noir/bin/noir in '_start'
  from ???

Versions

• OS: BlackArch
• Version: 0.19.0

Additional context
I tested all kind of different models, llama3.2, llama3.1, phi4, deepseek-chat.

[schniggie]

I had a short look at the source code (first time crytal :D). My first guess would be, the prompt forcing it to prodcuce JSON is not sufficient. I had a quick look at the Ollama docs and it seems there is a format option to force JSON too. So my first wild guess would be:

src/llm/ollama/ollama.cr:

Add:

:format => "json",

to

    def request(prompt : String)
      body = {
        :model  => @model,
        :prompt => prompt,
        :stream => false,
      }

[schniggie]

Ok did a short test with the suggested changes from above. Feels this goes further, but still fails with a new casting exception.

/home/xxx/noir/bin/noir -b .  -T --ollama http://192.168.178.250:11434 --ollama-model phi4:latest -u https://www.xxx.app

░█▄─░█ ░█▀▀▀█ ▀█▀ ░█▀▀█
░█░█░█ ░█──░█ ░█─ ░█▄▄▀
░█──▀█ ░█▄▄▄█ ▄█▄ ░█─░█ {v0.19.0}

⚑ Detecting technologies to base directory.
▲ No technologies detected.
  ➔ If you know the technology, use the -t flag to specify it.
  ➔ Please check tech lists using the --list-techs flag.
⚑ Start file-based analysis as the -u flag has been used.
⚑ Initializing analyzers
⚑ Analysis Started
  ➔ Code Analyzer: 0 in use
  ➔ AI Analyzer: Ollama in use
Unhandled exception: Cast from Hash(String, JSON::Any) to Array(T) failed, at /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/any.cr:260:5:260 (TypeCastError)
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/json/any.cr:259:3 in 'as_a'
  from /home/xxx/noir/src/analyzer/analyzers/llm_analyzers/ollama.cr:43:9 in 'analyze'
  from /home/xxx/noir/src/analyzer/analyzer.cr:18:3 in '->'
  from /home/xxx/noir/src/analyzer/analyzer.cr:75:38 in 'analysis_endpoints'
  from /home/xxx/noir/src/models/noir.cr:99:18 in 'analyze'
  from /home/xxx/noir/src/noir.cr:125:1 in '__crystal_main'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/crystal/main.cr:118:5 in 'main_user_code'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/crystal/main.cr:104:7 in 'main'
  from /home/linuxbrew/.linuxbrew/Cellar/crystal/1.15.0/share/crystal/src/crystal/system/unix/main.cr:9:3 in 'main'
  from /usr/lib/libc.so.6 in '??'
  from /usr/lib/libc.so.6 in '__libc_start_main'
  from /home/xxx/noir/bin/noir in '_start'
  from ???

[ksg97031]

Hi @schniggie,

Thank you for pointing out the format option!

I’m currently testing the AI integration code in the issue-518-ollama-json-format branch to explore potential improvements.
It seems we’ll need to conduct extensive testing across multiple projects to ensure its stability and effectiveness.

If you have any PRs to suggest or improvements to propose, please feel free to share them. We’d be happy to review them positively.

Thank you again for bringing this issue to our attention!