Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(ai-prompr-decorator): fix unable to modify request #13966

Merged
merged 3 commits into from
Dec 3, 2024
+213 −64
Merged

fix(ai-prompr-decorator): fix unable to modify request #13966

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
78d30c7
chore(ai): remove unused debug line
oowl Dec 2, 2024
1794697
fix(ai-prompr-decorator): fix unable to modify request
fffonion Dec 2, 2024
5434959
fix: prompt decorator crashes AI Gateway when trying to decorate "pre…
tysoekong Dec 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions kong/llm/plugin/shared-filters/serialize-analytics.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,6 @@ function _M:run(conf)
total_tokens = ai_plugin_o11y.metrics_get("llm_total_tokens_count"),
cost = ai_plugin_o11y.metrics_get("llm_usage_cost"),
}

kong.log.inspect(usage)

kong.log.set_serialize_value(string.format("ai.%s.usage", ai_plugin_o11y.NAMESPACE), usage)


Expand All @@ -82,4 +79,4 @@ function _M:run(conf)
return true
end

return _M
return _M
6 changes: 5 additions & 1 deletion kong/plugins/ai-prompt-decorator/filters/decorate-prompt.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
-- [ END OF LICENSE 0867164ffc95e54f04670b5169c09574bdbd9bba ]

local new_tab = require("table.new")
local deep_copy = require("kong.tools.table").deep_copy
local ai_plugin_ctx = require("kong.llm.plugin.ctx")

local _M = {
Expand Down Expand Up @@ -36,6 +37,9 @@ local function execute(request, conf)
local prepend = conf.prompts.prepend or EMPTY
local append = conf.prompts.append or EMPTY

-- ensure we don't modify the original request
request = deep_copy(request)

local old_messages = request.messages
local new_messages = new_tab(#append + #prepend + #old_messages, 0)
request.messages = new_messages
Expand Down Expand Up @@ -84,4 +88,4 @@ function _M:run(conf)
return true
end

return _M
return _M
Loading