[DOC] Change documentation for Rerank processor, add special case of Normalization processor #7392
Closed
1 of 4 tasks
Labels
2 - In progress
Issue/PR: The issue or PR is in progress.
What do you want to do?
Tell us about your request. Provide a summary of the request and all versions that are affected.
Existing documentation for Rerank processor should be enhanced, we need to add a special case when Rerank and Normalization processors are working together. In such case scores of Normalization processor will be updated by Rerank, and that may be an unexpected behavior for end user.
This behavior is not related to any specific version and was there from the moment of release of both processors. Ideally this change should be added as soon as possible, this behavior is there for users of 2.12 and above.
Suggested change:
Special Scenario: Interaction with Normalization Processor and Hybrid Query
When using the rerank processor in conjunction with the normalization processor and a hybrid query, there is an important behavior to be aware of: the rerank processor will alter the result scores of the documents. This is due to the fact that the rerank processor operates after the normalization processor in the search pipeline.
Understanding the Order of Processing
Normalization Processor: This processor normalizes the scores of the documents based on the configured normalization method. For more details on how this processor works, refer to the Normalization Processor documentation.
Rerank Processor: Following the normalization, the rerank processor will further adjust the scores of the documents. This adjustment can significantly impact the final ordering of search results.
Implications
What other resources are available? Provide links to related issues, POCs, steps for testing, etc.
Rerank processor page: https://opensearch.org/docs/latest/search-plugins/search-relevance/reranking-search-results/
Normalization processor page: https://opensearch.org/docs/latest/search-plugins/search-pipelines/normalization-processor/
The text was updated successfully, but these errors were encountered: