setup llama_cpp for direct inference (support MacOS)

[rgbkrk]

This sets up functionary on llama_cpp in a reusable way, at least for MacOS.

• The GGUF file for 7b-v2.2 gets downloaded from HuggingFace and cached for repeated runs
• Inference is set up to take messages and tools in the OpenAI format
• A FunctionRegistry from ChatLab can accept any functions with valid docstrings and types to make function declaration simple
• An example call asking for the weather in multiple locations is requested
• Multiple tool calls are handled via chatlab and returned back to the model for final inference
• The model is able to synthesize information from all of the function calls to provide a final response for the user

A few liberties on my local setup:

• My local environment is using Pydantic v2, as that's what is baked into ChatLab. I don't see anything in the current functionary code requiring it to stay on v1. I'd love to update that in the requirements.txt
• termcolor is required for this example script as well as chatlab 1.3.0. These could be added into requirements.txt.

[musabgultekin]

Thank you for the PR!

It looks good to me, maybe we can move this example to an examples topfolder.

[musabgultekin]

Also, I think there was a small problem with pydantic v2. I cannot remember exactly. @khai-meetkai mentioned earlier on.

[rgbkrk]

Thank you for the PR!

It looks good to me, maybe we can move this example to an examples topfolder.

I'm happy to make this into a server as well since you've got all the definitions here. Relatedly, do you plan to release the functionary Python code as a PyPI package so that it can be used directly without cloning the repository?

Have you all considered adding this model to Ollama? I'd love to get it in more people's hands.

[rgbkrk]

In order to add this example to an examples folder, I need it to be able to import functionary. I made a separate PR for the packaging aspect: #95

[rgbkrk]

Verifying this works with the v2.2 model now.

[shreyaskarnik]

This is awesome @rgbkrk! Unblocks a lot of macOS users. An addition to Ollama would be amazing!

[shreyaskarnik]

Following this example I was able to demonstrate how to use langchain tools with functionary https://gist.github.com/shreyaskarnik/2cc099528f14671b096570498330ae54 there may be a better way to handle the actual function execution and I am open to learning the right way to do so.