feat: add Max Tokens and Context Window Setting Options for Ollama Channel #1694
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MotorBottle
changed the title
|
本地测了一下挺好的 建议合并 |
c121914yu
approved these changes
Aug 6, 2024
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1694 +/- ##
========================================
- Coverage 1.30% 1.30% -0.01%
========================================
Files 144 144
Lines 10140 10143 +3
========================================
Hits 132 132
- Misses 9994 9997 +3
Partials 14 14 ☔ View full report in Codecov by Sentry. |
songquanpeng
changed the title
|
Thx~ |
mxdlzg
pushed a commit
to mxdlzg/one-api
that referenced
this pull request
Oct 15, 2024
…annel (songquanpeng#1694) * Update main.go with max_tokens param * Update model.go with max_tokens param * Update model.go * Update main.go * Update main.go * Adds num_ctx param for Ollama Channel * Added num_ctx param for ollama adapter * Added num_ctx param for ollama adapter * Improved data process logic
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
已实现：
max_tokens（对应ollama的num_predict）参数限制输出token数num_ctx（ollama原生参数）参数设定更改ollama默认context window大小（默认仅1k/2k上下文）有个问题：
下图中改动的部分是因为原本的代码会导致报错，console打印后发现原本的data就是正确的json格式不需要对花括号进行增减处理。但奇怪的是目前我通过docker compose部署的正式版本也是能正常使用的，正式版本并没有我对这两行的修改（也可能部署的版本本来没有这两行？没有去查看，总之修改之后添加了条件判断，对两种不同状况都能有效应对）。
目前已确认并非个案， #1702 也是同样的问题，但我觉得条件判断会比直接改掉健壮性更强
close #1691
我已确认该 PR 已自测通过，相关截图如下：
