-
-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Anthropic prompt caching #104
base: main
Are you sure you want to change the base?
Conversation
✅ Deploy Preview for stunning-cannoli-238aa5 canceled.
|
|
||
enum AnthropicCacheType | ||
{ | ||
case ethemeral; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see value of keeping the enum here but Imo keeping provider related settings in the prisms code can make maintenance way harder.
@sixlive can think different.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hilariously, having added the enum to avoid spelling mistakes, I misspelt it in the enum...! Now fixed.
Definitely not a huge loss if we don't include the enum. Really just a convenience thing.
Implements Anthropic prompt caching
Choices made
providerMeta: [Provider::Anthropic->value -> [...]]
. It ended up easy enough to do with a trait, shared with Tool.I don't think providerMeta is the nicest API for the user, and it might get messy in the map methods if more features become reliant on it. But it should be easy to add a nicer API in a backward compatible way in future if needed. My gut thought is a ->withFeature() method on Messages and Tools, which accepts an implementation of a contract with a handle method, which are then applied with a pipeline to mutate the payload.
To-do
P.S. could do away with the
AnthropicCacheType
enum - however if anyone else is like me that can never spell "ephemeral" right - it is quite handy!