Allow for arguments to be passed to the base model

[havietisov]

looking at auto-gptq code I spotted that it already contains kv-caching :
https://github.com/PanQiWei/AutoGPTQ/blob/main/auto_gptq/nn_modules/fused_llama_attn.py

would be really nice to let users utilize this.

[rlouf]

I agree. #190 tracks the discussion around KV caching. We have bigger plans for this, but we've been focusing on constrained generation almost exclusively so far 🙂

We can certainly implement the naive version quickly, it's just a small interface change. Will bump it up on the priority list.

[havietisov]

I could borrow the mechanism from huggingface transformers if not for #209, which makes the process of installing from source a bit sketchy.

[rlouf]

I'm going to investigate this one more tomorrow, may have some follow up questions.