From 797630abdf5381e5617ec1547a90330d6a917d50 Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Sun, 9 Apr 2023 01:29:46 +0600 Subject: [PATCH 1/8] Create advanced index actions guide Signed-off-by: Alexei Karikov --- guides/advanced_index_actions.md | 105 +++++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 guides/advanced_index_actions.md diff --git a/guides/advanced_index_actions.md b/guides/advanced_index_actions.md new file mode 100644 index 00000000..4308b190 --- /dev/null +++ b/guides/advanced_index_actions.md @@ -0,0 +1,105 @@ +# Advanced Index Actions + +In this guide, we will look at some advanced index actions that are not covered in the [Index Lifecycle](index_lifecycle.md) guide. + +## Setup + +Let's create a client instance, and an index named `movies`: + +```python +from opensearchpy import OpenSearch +client = OpenSearch( + hosts=['https://admin:admin@localhost:9200'], + use_ssl=True, + verify_certs=False +) +client.indices.create(index='movies') +``` + +## API Actions + +### Clear index cache + +You can clear the cache of an index or indices by using the `indices.clear_cache` API action. The following example clears the cache of the `movies` index: + +```python +client.indices.clear_cache(index='movies') +``` + +By default, the `indices.clear_cache` API action clears all types of cache. To clear specific types of cache pass the the `query`, `fielddata`, or `request` parameter to the API action: + +```python +client.indices.clear_cache(index='movies', query=True) +client.indices.clear_cache(index='movies', fielddata=True, request=True) +``` + +### Flush index + +Sometimes you might want to flush an index or indices to make sure that all data in the transaction log is persisted to the index. To flush an index or indices use the `indices.flush` API action. The following example flushes the `movies` index: + +```python +client.indices.flush(index='movies') +``` + +### Refresh index + +You can refresh an index or indices to make sure that all changes are available for search. To refresh an index or indices use the `indices.refresh` API action: + +```python +client.indices.refresh(index='movies') +``` + +### Open/Close index + +You can close an index to prevent read and write operations on the index. A closed index does not have to maintain certain data structures that an opened index require, reducing the memory and disk space required by the index. The following example closes and reopens the `movies` index: + +```python +client.indices.close(index='movies') +client.indices.open(index='movies') +``` + +### Force merge index + +You can force merge an index or indices to reduce the number of segments in the index. This can be useful if you have a large number of small segments in the index. Merging segments reduces the memory footprint of the index. Do note that this action is resource intensive and it is only recommended for read-only indices. The following example force merges the `movies` index: + +```python +client.indices.forcemerge(index='movies') +``` + +### Clone index + +You can clone an index to create a new index with the same mappings, data, and MOST of the settings. The source index must be in read-only state for cloning. The following example blocks write operations from `movies` index, clones the said index to create a new index named `movies_clone`, then re-enables write: + +```python +client.indices.put_settings(index='movies', body={'index': {'blocks': {'write': True}}}) +client.indices.clone(index='movies', target='movies_clone') +client.indices.put_settings(index='movies', body={'index': {'blocks': {'write': False}}}) +``` + +### Split index + +You can split an index into another index with more primary shards. The source index must be in read-only state for splitting. The following example create the read-only `books` index with 30 routing shards and 5 shards (which is divisible by 30), splits index into `bigger_books` with 10 shards (which is also divisible by 30), then re-enables write: + +```python +client.indices.create( + index='books', + body={ 'settings': { + 'index': { 'number_of_shards': 5, + 'number_of_routing_shards': 30, + 'blocks': { 'write': True } } } }) + +client.indices.split( + index='books', + target='bigger_books', + body={ 'settings': { 'index': { 'number_of_shards': 10 } } }) + +client.indices.put_settings(index='books', body={ 'index': { 'blocks': { 'write': False } } }) +``` + +## Cleanup + +Let's delete all the indices we created in this guide: + +```python +client.indices.delete(index=['movies', 'books', 'movies_clone', 'bigger_books']) +``` From c885074b3f64c2fa1a6eba7cde5dcc53331a7ab8 Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Thu, 20 Apr 2023 00:45:06 +0600 Subject: [PATCH 2/8] Add changelog entry Signed-off-by: Alexei Karikov --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b38fd86b..0a558919 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ## [Unreleased] ### Added - Added async support for helpers that are merged from opensearch-dsl-py ([#329](https://github.com/opensearch-project/opensearch-py/pull/329)) +- Added advanced index actions guide ([#363](https://github.com/opensearch-project/opensearch-py/pull/363)) ### Changed - Upgrading pytest-asyncio to latest version - 0.21.0 ([#339](https://github.com/opensearch-project/opensearch-py/pull/339)) ### Deprecated From a05540073ffa1cb98cd38b1a735bd344259cd072 Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Tue, 5 Sep 2023 18:31:56 +0600 Subject: [PATCH 3/8] Add changelog entry Signed-off-by: Alexei Karikov --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 040fb8ab..0d6f44ca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ## [Unreleased] ### Added - Added generating imports and headers to API generator ([#467](https://github.com/opensearch-project/opensearch-py/pull/467)) +- Added advanced index actions guide ([#363](https://github.com/opensearch-project/opensearch-py/pull/363)) ### Changed ### Deprecated ### Removed From 764686fea9327917a9d1cdf8c296955e1b6d24c7 Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Tue, 5 Sep 2023 18:35:52 +0600 Subject: [PATCH 4/8] Add link from users guide Signed-off-by: Alexei Karikov --- USER_GUIDE.md | 1 + 1 file changed, 1 insertion(+) diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 4b6b89c2..9a722081 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -153,6 +153,7 @@ print(response) - [Search](guides/search.md) - [Point in Time](guides/point_in_time.md) - [Using a Proxy](guides/proxy.md) +- [Advanced index actions](guides/advanced_index_actions.md) ## Plugins From 170f07692b2c570247737d4e817a93ea7929eb00 Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Tue, 5 Sep 2023 21:12:18 +0600 Subject: [PATCH 5/8] Add a TOC and capitalize anchors Signed-off-by: Alexei Karikov --- guides/advanced_index_actions.md | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/guides/advanced_index_actions.md b/guides/advanced_index_actions.md index 4308b190..94b70e6d 100644 --- a/guides/advanced_index_actions.md +++ b/guides/advanced_index_actions.md @@ -1,3 +1,15 @@ +# Advanced Index Actions Guide +- [Advanced Index Actions](#advanced-index-actions) + - [Setup](#setup) + - [Api Actions](#api-actions) + - [Clear Index Cache](#clear-index-cache) + - [Flush Index](#flush-index) + - [Refresh Index](#refresh-index) + - [Open/Close Index](#open-close-index) + - [Force merge index](#force-merge-index) + - [Clone index](#clone-index) + - [Split index](#split-index) + - [Cleanup](#cleanup) # Advanced Index Actions In this guide, we will look at some advanced index actions that are not covered in the [Index Lifecycle](index_lifecycle.md) guide. @@ -18,7 +30,7 @@ client.indices.create(index='movies') ## API Actions -### Clear index cache +### Clear Index Cache You can clear the cache of an index or indices by using the `indices.clear_cache` API action. The following example clears the cache of the `movies` index: @@ -33,7 +45,7 @@ client.indices.clear_cache(index='movies', query=True) client.indices.clear_cache(index='movies', fielddata=True, request=True) ``` -### Flush index +### Flush Index Sometimes you might want to flush an index or indices to make sure that all data in the transaction log is persisted to the index. To flush an index or indices use the `indices.flush` API action. The following example flushes the `movies` index: @@ -41,7 +53,7 @@ Sometimes you might want to flush an index or indices to make sure that all data client.indices.flush(index='movies') ``` -### Refresh index +### Refresh Index You can refresh an index or indices to make sure that all changes are available for search. To refresh an index or indices use the `indices.refresh` API action: @@ -49,7 +61,7 @@ You can refresh an index or indices to make sure that all changes are available client.indices.refresh(index='movies') ``` -### Open/Close index +### Open/Close Index You can close an index to prevent read and write operations on the index. A closed index does not have to maintain certain data structures that an opened index require, reducing the memory and disk space required by the index. The following example closes and reopens the `movies` index: From aa842bb84f6822bdf34175cb88f178d2486ff5bd Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Tue, 5 Sep 2023 21:17:12 +0600 Subject: [PATCH 6/8] Capitalize Index Actions Signed-off-by: Alexei Karikov --- USER_GUIDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 9a722081..ed9ea603 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -153,7 +153,7 @@ print(response) - [Search](guides/search.md) - [Point in Time](guides/point_in_time.md) - [Using a Proxy](guides/proxy.md) -- [Advanced index actions](guides/advanced_index_actions.md) +- [Advanced Index Actions](guides/advanced_index_actions.md) ## Plugins From 326425463c9712ec0675a6af798e81151218795f Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Tue, 5 Sep 2023 23:02:17 +0600 Subject: [PATCH 7/8] Capitalize Split index, Clone index, Force merge index Signed-off-by: Alexei Karikov --- guides/advanced_index_actions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/guides/advanced_index_actions.md b/guides/advanced_index_actions.md index 94b70e6d..37c5a4bc 100644 --- a/guides/advanced_index_actions.md +++ b/guides/advanced_index_actions.md @@ -6,9 +6,9 @@ - [Flush Index](#flush-index) - [Refresh Index](#refresh-index) - [Open/Close Index](#open-close-index) - - [Force merge index](#force-merge-index) - - [Clone index](#clone-index) - - [Split index](#split-index) + - [Force Merge Index](#force-merge-index) + - [Clone Index](#clone-index) + - [Split Index](#split-index) - [Cleanup](#cleanup) # Advanced Index Actions From 2e0a08d405f067a9189a4cc12b157474ea370461 Mon Sep 17 00:00:00 2001 From: Alexei Karikov Date: Tue, 5 Sep 2023 23:29:07 +0600 Subject: [PATCH 8/8] Capitalize Split index, Clone index, Force merge index Signed-off-by: Alexei Karikov --- guides/advanced_index_actions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/guides/advanced_index_actions.md b/guides/advanced_index_actions.md index 37c5a4bc..b049470d 100644 --- a/guides/advanced_index_actions.md +++ b/guides/advanced_index_actions.md @@ -70,7 +70,7 @@ client.indices.close(index='movies') client.indices.open(index='movies') ``` -### Force merge index +### Force Merge Index You can force merge an index or indices to reduce the number of segments in the index. This can be useful if you have a large number of small segments in the index. Merging segments reduces the memory footprint of the index. Do note that this action is resource intensive and it is only recommended for read-only indices. The following example force merges the `movies` index: @@ -78,7 +78,7 @@ You can force merge an index or indices to reduce the number of segments in the client.indices.forcemerge(index='movies') ``` -### Clone index +### Clone Index You can clone an index to create a new index with the same mappings, data, and MOST of the settings. The source index must be in read-only state for cloning. The following example blocks write operations from `movies` index, clones the said index to create a new index named `movies_clone`, then re-enables write: @@ -88,7 +88,7 @@ client.indices.clone(index='movies', target='movies_clone') client.indices.put_settings(index='movies', body={'index': {'blocks': {'write': False}}}) ``` -### Split index +### Split Index You can split an index into another index with more primary shards. The source index must be in read-only state for splitting. The following example create the read-only `books` index with 30 routing shards and 5 shards (which is divisible by 30), splits index into `bigger_books` with 10 shards (which is also divisible by 30), then re-enables write: