Consideration - Compress LLM template/query optimization

[converseKarl]

there is another git project from Microsoft for LLM Prompt Query compression thatr reduces query prompt by 20% and optimizes it but also reduces the LLM costing and speeds up performance.

https://github.com/microsoft/LLMLingua

Could this be something useful to hook in as an optimization on the LLM template and query?

[adhityan]

Yes, this is certainly useful.

I checked this out when they put it out. I want to enable it as a simple flag you set to activate during build. Unfortunately Microsoft has only provided a Python implementation for this library atm. I am keeping track of this and we can reopen this thread once there is support for it in JS/TS.