Skip to content

Commit

Permalink
Added new guide & sample module for using index templates.
Browse files Browse the repository at this point in the history 
Signed-off-by: Djcarrillo6 <[email protected]>

Fixed line wrapping & connect params in index_template.md

Signed-off-by: Djcarrillo6 <[email protected]>

Linked sample code to index_template.md file

Signed-off-by: Djcarrillo6 <[email protected]>

Fixed broken link in to sample in index_template.md

Signed-off-by: Djcarrillo6 <[email protected]>

Added http_auth param to the index_template_sample.py module

Signed-off-by: Djcarrillo6 <[email protected]>

Fixed MD files.

Signed-off-by: Djcarrillo6 <[email protected]>

Fixed MD files again

Signed-off-by: Djcarrillo6 <[email protected]>

Fixed MD files again 2

Signed-off-by: Djcarrillo6 <[email protected]>
  • Loading branch information
@Djcarrillo6
Djcarrillo6 committed Oct 11, 2023
1 parent ce835fb commit b450075
Show file tree
Hide file tree
Showing 4 changed files with 329 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
### Added
- Added generating imports and headers to API generator ([#467](https://github.com/opensearch-project/opensearch-py/pull/467))
- Added point-in-time APIs (create_pit, delete_pit, delete_all_pits, get_all_pits) and Security Client APIs (health and update_audit_configuration) ([#502](https://github.com/opensearch-project/opensearch-py/pull/502))
- Added new guide for using index templates with the client ([#531](https://github.com/opensearch-project/opensearch-py/pull/531))
### Changed
- Generate `tasks` client from API specs ([#508](https://github.com/opensearch-project/opensearch-py/pull/508))
- Generate `ingest` client from API specs ([#513](https://github.com/opensearch-project/opensearch-py/pull/513))
Expand Down
1 change: 1 addition & 0 deletions USER_GUIDE.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ print(response)
- [Search](guides/search.md)
- [Point in Time](guides/point_in_time.md)
- [Using a Proxy](guides/proxy.md)
- [Index Templates](guides/index_template.md)

## Plugins

Expand Down
184 changes: 184 additions & 0 deletions guides/index_template.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
# Index Template
Index templates are a convenient way to define settings, mappings, and aliases for one or more indices when they are created. In this guide, you'll learn how to create an index template and apply it to an index.

## Setup

Assuming you have OpenSearch running locally on port 9200, you can create a client instance with the following code:
```python
from opensearchpy import OpenSearch
client = OpenSearch(
hosts=['https://localhost:9200'],
use_ssl=True,
verify_certs=False,
http_auth=('admin', 'admin')
)
```

## Index Template API Actions

### Create an Index Template
You can create an index template to define default settings and mappings for indices of certain patterns. The following example creates an index template named `books` with default settings and mappings for indices of the `books-*` pattern:

```python
client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
},
'mappings': {
'properties': {
'title': { 'type': 'text' },
'author': { 'type': 'text' },
'published_on': { 'type': 'date' },
'pages': { 'type': 'integer' }
}
}
}
}
)
```

Now, when you create an index that matches the `books-*` pattern, OpenSearch will automatically apply the template's settings and mappings to the index. Let's create an index named `books-nonfiction` and verify that its settings and mappings match those of the template:

```python
client.indices.create(index='books-nonfiction')
print(client.indices.get(index='books-nonfiction'))
```

### Multiple Index Templates
If multiple index templates match the index's name, OpenSearch will apply the template with the highest priority. The following example creates two index templates named `books-*` and `books-fiction-*` with different settings:

```python
client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'priority': 0, # default priority
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
}
}
}
)

client.indices.put_index_template(
name='books-fiction',
body={
'index_patterns': ['books-fiction-*'],
'priority': 1, # higher priority than the `books` template
'template': {
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 1
}
}
}
}
)
```

When we create an index named `books-fiction-romance`, OpenSearch will apply the `books-fiction-*` template's settings to the index:

```python
client.indices.create(index='books-fiction-romance')
print(client.indices.get(index='books-fiction-romance'))
```

### Composable Index Templates
Composable index templates are a new type of index template that allow you to define multiple component templates and compose them into a final template. The following example creates a component template named `books_mappings` with default mappings for indices of the `books-*` and `books-fiction-*` patterns:

```python
client.cluster.put_component_template(
name='books_mappings',
body={
'template': {
'mappings': {
'properties': {
'title': { 'type': 'text' },
'author': { 'type': 'text' },
'published_on': { 'type': 'date' },
'pages': { 'type': 'integer' }
}
}
}
}
)

client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'composed_of': ['books_mappings'], # use the `books_mappings` component template
'priority': 0,
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
}
}
}
)

client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'composed_of': ['books_mappings'], # use the `books_mappings` component template
'priority': 1,
'template': {
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 1
}
}
}
}
)
```

When we create an index named `books-fiction-horror`, OpenSearch will apply the `books-fiction-*` template's settings, and `books_mappings` template mappings to the index:

```python
client.indices.create(index='books-fiction-horror')
print(client.indices.get(index='books-fiction-horror'))
```

### Get an Index Template
You can get an index template with the `get_index_template` API action:

```python
print(client.indices.get_index_template(name='books'))
```

### Delete an Index Template
You can delete an index template with the `delete_template` API action:

```python
client.indices.delete_index_template(name='books')
```

## Cleanup
Let's delete all resources created in this guide:

```python
client.indices.delete(index='books-*')
client.indices.delete_index_template(name='books-fiction')
client.cluster.delete_component_template(name='books_mappings')
```

# Sample Code
See [index_template_sample.py](/samples/index_template/index_template_sample.py) for a working sample of the concepts in this guide.
143 changes: 143 additions & 0 deletions samples/index_template/index_template_sample.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
from opensearchpy import OpenSearch

# Create a client instance
client = OpenSearch(
hosts=['https://localhost:9200'],
use_ssl=True,
verify_certs=False,
http_auth=('admin', 'admin')
)

# You can create an index template to define default settings and mappings for indices of certain patterns. The following example creates an index template named `books` with default settings and mappings for indices of the `books-*` pattern:
client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'priority': 1,
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
},
'mappings': {
'properties': {
'title': { 'type': 'text' },
'author': { 'type': 'text' },
'published_on': { 'type': 'date' },
'pages': { 'type': 'integer' }
}
}
}
}
)

# Now, when you create an index that matches the `books-*` pattern, OpenSearch will automatically apply the template's settings and mappings to the index. Let's create an index named books-nonfiction and verify that its settings and mappings match those of the template:
client.indices.create(index='books-nonfiction')
print(client.indices.get(index='books-nonfiction'))

# If multiple index templates match the index's name, OpenSearch will apply the template with the highest `priority`. The following example creates two index templates named `books-*` and `books-fiction-*` with different settings:
client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'priority': 1,
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
}
}
}
)

client.indices.put_index_template(
name='books-fiction',
body={
'index_patterns': ['books-fiction-*'],
'priority': 2,
'template': {
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 1
}
}
}
}
)

# # Test multiple index templates
client.indices.create(index='books-fiction-romance')
print(client.indices.get(index='books-fiction-romance'))


# Composable index templates are a new type of index template that allow you to define multiple component templates and compose them into a final template. The following example creates a component template named `books_mappings` with default mappings for indices of the `books-*` and `books-fiction-*` patterns:
client.cluster.put_component_template(
name='books_mappings',
body={
'template': {
'mappings': {
'properties': {
'title': { 'type': 'text' },
'author': { 'type': 'text' },
'published_on': { 'type': 'date' },
'pages': { 'type': 'integer' }
}
}
}
}
)

client.indices.put_index_template(
name='books',
body={
'index_patterns': ['books-*'],
'composed_of': ['books_mappings'],
'priority': 4,
'template': {
'settings': {
'index': {
'number_of_shards': 3,
'number_of_replicas': 0
}
}
}
}
)

client.indices.put_index_template(
name='books-fiction',
body={
'index_patterns': ['books-fiction-*'],
'composed_of': ['books_mappings'],
'priority': 5,
'template': {
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 1
}
}
}
}
)


# Test composable index templates
client.indices.create(index='books-fiction-horror')
print(client.indices.get(index='books-fiction-horror'))

# Get an index template
print(client.indices.get_index_template(name='books'))

# Delete an index template
client.indices.delete_index_template(name='books')

# Cleanup
client.indices.delete(index='books-*')
client.indices.delete_index_template(name='books-fiction')
client.cluster.delete_component_template(name='books_mappings')

0 comments on commit b450075

Please sign in to comment.