Feat/corcel+llama+gemini

[dvilelaf]

Proposed changes

Adds corcel, llama and gemini request tools that support both prediction and completion.

Fixes

N/A

Types of changes

What types of changes does your code introduce? (A breaking change is a fix or feature that would cause existing functionality and APIs to not work as expected.)
Put an x in the box that applies

• Non-breaking fix (non-breaking change which fixes an issue)
• Breaking fix (breaking change which fixes an issue)
• Non-breaking feature (non-breaking change which adds functionality)
• Breaking feature (breaking change which adds functionality)
• Refactor (non-breaking change which changes implementation)
• Messy (mixture of the above - requires explanation!)

Checklist

Put an x in the boxes that apply.

• I have read the CONTRIBUTING doc
• I am making a pull request against the main branch (left side). Also you should start your branch off our main.
• Lint and unit tests pass locally with my changes
• I have added tests that prove my fix is effective or that my feature works

Further comments

N/A

[dvilelaf]

@0xArdi this tool runs small LLM models on the CPU. When this command is run for the first time, it downloads the model (~4GB) to a local directory, which means that the first execution could take 2-3 minutes depending on the connection. Would this be a problem for the mech?

Also, inference can take up to 1 minute depending on the prompts and uses a lot of CPU and RAM.

[0xArdi]

We have had issues before with models in memory, as they can lead to spikes to resource usage resulting in the container being killed by k8s. So unless this is required to be deployed, I'd say let's not deploy this tool.

That being said, with the introduction of mech-marketplace, what I said above doesn't hold, and if someone can chose to run it.

[dvilelaf]

Ok, let's not deploy it then for now, but I guess we can merge it. What's that mech marketplace thing?

[dvilelaf]

Not sure what is happening to Windows lock check. It seems to stall while installing tomte.

[dvilelaf]

@0xArdi I'm getting a linter error failure on an already existing component:

Processing /home/david/Valory/repos/mech/packages/nickcom007/customs/prediction_request_sme_lite/component.yaml
Failed!
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='w' encoding='UTF-8'>
ERROR: InvocationError for command /home/david/Valory/repos/mech/.tox/check-packages/bin/autonomy check-packages (exited with code 1)
__________________________________________________________________________________________ summary ___________________________________________________________________________________________
ERROR:   check-packages: commands failed