Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Clarify that text embedding model will do truncation #7365

Open
1 of 4 tasks
ylwu-amzn opened this issue Jun 11, 2024 · 2 comments
Open
1 of 4 tasks

[DOC] Clarify that text embedding model will do truncation #7365

ylwu-amzn opened this issue Jun 11, 2024 · 2 comments
Assignees
Labels
1 - Backlog - DEV Developer assigned to issue is responsible for creating PR.

Comments

@ylwu-amzn
Copy link
Contributor

What do you want to do?

  • Request a change to existing documentation
  • Add new documentation
  • Report a technical problem with the documentation
  • Other

Tell us about your request. Provide a summary of the request and all versions that are affected.

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

@ylwu-amzn
Copy link
Contributor Author

Read more details opensearch-project/ml-commons#2466

the trouble here is not whether it's configurable its about user understanding. If someone is new to vector search this is a really easy way to shoot yourself in the foot and never know what went wrong. I've talked to several people already who have completely abandoned vector search because they thought the relevancy was bad. Turns out their documents were just truncated because they didn't understand how vectorization worked.

@kolchfa-aws
Copy link
Collaborator

@dtaivpp Could you raise a PR for this since you are familiar with this issue?

@hdhalter hdhalter added 1 - Backlog - DEV Developer assigned to issue is responsible for creating PR. and removed 1 - Backlog - DOC Doc writer assigned to issue responsible for creating PR. labels Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1 - Backlog - DEV Developer assigned to issue is responsible for creating PR.
Projects
None yet
Development

No branches or pull requests

3 participants