Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the term hybrid search in the semantic tutorial #5103

Merged
merged 2 commits into from
Sep 25, 2023
Merged

Use the term hybrid search in the semantic tutorial #5103

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a549526
Use the term hybrid search in the semantic tutorial
kolchfa-aws Sep 25, 2023
1742e31
Fix link
kolchfa-aws Sep 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions _ml-commons-plugin/semantic-search.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
In this tutorial, you'll learn how to:

- Implement semantic search in OpenSearch.
- Combine semantic search with keyword search to improve search relevance.
- Implement hybrid search by combining semantic and keyword search to improve search relevance.

## Terminology

Expand All @@ -26,6 +26,7 @@
- At search time, when you then use a _neural query_, the query text is passed through a language model, and the resulting vector embeddings are compared with the document text vector embeddings to find the most relevant results, as shown in the following diagram.

![Neural search at search time diagram]({{site.url}}{{site.baseurl}}/images/neural-search-query.png)
- _Hybrid search_: Combines semantic and keyword search to improve search relevance.

## OpenSearch components for semantic search

Expand Down Expand Up @@ -87,7 +88,7 @@
1. [**Search the data**](#step-3-search-the-data).
- [Search using a keyword search](#search-using-a-keyword-search).
- [Search using a neural search](#search-using-a-neural-search).
- [Search using a combined keyword search and neural search](#search-using-a-combined-keyword-search-and-neural-search).
- [Search using a hybrid search](#search-using-a-hybrid-search).

Some steps in the tutorial contain optional `Test it` sections. You can ensure that the step was successful by running requests in these sections.

Expand Down Expand Up @@ -820,9 +821,9 @@
```
</details>

### Search using a combined keyword search and neural search
### Search using a hybrid search

To combine keyword search and neural search, you need to set up a [search pipeline]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/index/) that runs at search time. The search pipeline you'll configure intercepts search results at an intermediate stage and applies the [`normalization_processor`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/normalization-processor/) to them. The `normalization_processor` normalizes and combines the document scores from multiple query clauses, rescoring the documents according to the chosen normalization and combination techniques.
Hybrid search combines keyword and neural search to improve search relevance. To implement hybrid search, you need to set up a [search pipeline]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/index/) that runs at search time. The search pipeline you'll configure intercepts search results at an intermediate stage and applies the [`normalization_processor`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/normalization-processor/) to them. The `normalization_processor` normalizes and combines the document scores from multiple query clauses, rescoring the documents according to the chosen normalization and combination techniques.

Check failure on line 826 in _ml-commons-plugin/semantic-search.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _ml-commons-plugin/semantic-search.md#L826 

[OpenSearch.Spelling] Error: rescoring. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.
Raw output 
{"message": "[OpenSearch.Spelling] Error: rescoring. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_ml-commons-plugin/semantic-search.md", "range": {"start": {"line": 826, "column": 572}}}, "severity": "ERROR"}

#### Step 1: Configure a search pipeline

Expand Down Expand Up @@ -986,7 +987,7 @@

### Clean up

After you're done, delete the components you've created in tutorial from the cluster:
After you're done, delete the components you've created in this tutorial from the cluster:

```json
DELETE /my-nlp-index
Expand Down
Loading