Fix/ tools #169
Merged
Fix/ tools #169
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
0xArdi
commented
Feb 19, 2024
| @@ -119,7 +119,7 @@ | |||
| * Never use Markdown syntax highlighting, such as ```json```. Only output the raw json string. | |||
| * This is incorrect:"```json{{\n \"queries\": [\"term1\", \"term2\"]}}```" | |||
| * This is incorrect:```json"{{\n \"queries\": [\"term1\", \"term2\"]}}"``` | |||
| * This is correct:"{{\n \"quries\": [\"term1\", \"term2\"]}}" | |||
| * This is correct:"{{\n \"queries\": [\"term1\", \"term2\"]}}" | |||
There was a problem hiding this comment.
typo, which mightve played a role in invalid responses from claude
0xArdi
commented
Feb 19, 2024
Comment on lines
+168
to
+188
| """Custom OpenAI embedding function""" | ||
|
|
||
| def __init__(self, *args: Any, **kwargs: Any) -> None: | ||
| """Initialize custom OpenAI embedding function""" | ||
| super().__init__(*args, **kwargs) | ||
| # OpenAI@v1 compatible | ||
| self._client = openai.embeddings | ||
|
|
||
| def __call__(self, texts: Documents) -> Embeddings: | ||
| """Return embedding""" | ||
| # replace newlines, which can negatively affect performance. | ||
| texts = [t.replace("\n", " ") for t in texts] | ||
|
|
||
| # Call the OpenAI Embedding API | ||
| embeddings = self._client.create(input=texts, model=self._model_name).data | ||
|
|
||
| # Sort resulting embeddings by index | ||
| sorted_embeddings = sorted(embeddings, key=lambda e: e.index) # type: ignore | ||
|
|
||
| # Return just the embeddings | ||
| return [result.embedding for result in sorted_embeddings] |
There was a problem hiding this comment.
Required as the default one is not compatible with openai@v1
0xArdi
commented
Feb 19, 2024
| # MODELS | ||
|
|
||
| MAX_TEXT_LENGTH = 7500 |
There was a problem hiding this comment.
@namesty we need to make sure the length of the text doesnt exceed the limitations of the model. Added this in here for now, please propose a more appropriate number / algorithm to avoid this if you think this is not ok.
angrybayblade
approved these changes
Feb 20, 2024
Adamantios
approved these changes
Feb 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes misc issues with tools.