Skip to content

Commit

Permalink
Apply suggestions from code review
Browse files Browse the repository at this point in the history
Co-authored-by: Nathan Bower <[email protected]>
Co-authored-by: Chris Moore <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
  • Loading branch information
3 people authored Sep 20, 2023
1 parent aab7675 commit 9535205
Show file tree
Hide file tree
Showing 7 changed files with 49 additions and 49 deletions.
6 changes: 3 additions & 3 deletions _benchmark/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,15 @@ OpenSearch Benchmark is a macrobenchmark utility provided by the [OpenSearch Pro

OpenSearch Benchmark can be installed directly on a compatible host running Linux and macOS. You can also run OpenSearch Benchmark in a Docker container. See [Installing OpenSearch Benchmark]({{site.url}}{{site.baseurl}}/benchmark/installing-benchmark/) for more information.

The following diagram helps visualize how OpenSearch Benchmark works when run against a local host:
The following diagram visualizes how OpenSearch Benchmark works when run against a local host:

![Benchmark workflow]({{site.url}}{{site.baseurl}}/images/benchmark/OSB-workflow.png).

The OpenSearch Benchmark documentation is split into five sections:

- [Quickstart]({{site.url}}{{site.baseurl}}/benchmark/quickstart/): Learn how to quickly run and install OpenSearch Benchmark.
- [User guide]({{site.url}}{{site.baseurl}}/benchmark/user-guide/index/): Dive deep into how OpenSearch Benchmark can you help you track the performance of your cluster.
- [Tutorials]({{site.url}}{{site.baseurl}}/benchmark/tutorials/index/): Use step-by-step guides for more advanced Benchmarking configurations and functionality.
- [User guide]({{site.url}}{{site.baseurl}}/benchmark/user-guide/index/): Dive deep into how OpenSearch Benchmark can help you track the performance of your cluster.
- [Tutorials]({{site.url}}{{site.baseurl}}/benchmark/tutorials/index/): Use step-by-step guides for more advanced benchmarking configurations and functionality.
- [Commands]({{site.url}}{{site.baseurl}}/benchmark/commands/index/): A detailed reference of commands and command options supported by OpenSearch.
- [Workloads]({{site.url}}{{site.baseurl}}/benchmark/workloads/index/): A detailed reference of options available for both default and custom workloads.

Expand Down
42 changes: 21 additions & 21 deletions _benchmark/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ nav_order: 2

# OpenSearch Benchmark quickstart

This tutorial outlines how to quickly install OpenSearch Benchmark (OSB) and run your first OpenSearch Benchmark workload.
This tutorial outlines how to quickly install OpenSearch Benchmark and run your first OpenSearch Benchmark workload.

## Prerequisites

Expand All @@ -15,16 +15,16 @@ To perform the Quickstart steps, you'll need to fulfill the following prerequisi
- A currently active OpenSearch cluster. For instructions on how to create an OpenSearch cluster, see [Creating a cluster]({{site.url}}{{site.baseurl}}//tuning-your-cluster/index/).
- Git 2.3 or greater.

Additional prerequisites are required depending on your installation method.
Additional prerequisites are required, depending on your installation method:

- If you plan to install OpenSearch Benchmark with [PyPi](https://pypi.org/), install Python 3.8 or greater.
- If you plan to install OpenSearch Benchmark with [PyPi](https://pypi.org/), install Python 3.8 or later.
- If you plan to install OpenSearch Benchmark using Docker, install Docker.

## Installing OpenSearch Benchmark

You can install OpenSearch Benchmark using either PyPi or Docker.

If you plan to run OpenSearch Benchmark with a cluster using AWS Signature Version 4 (AWS Sigv4), see [Sigv4 support]({{site.url}}{{site.baseurl}}/benchmark/tutorials/sigv4/).
If you plan to run OpenSearch Benchmark with a cluster using AWS Signature Version 4, see [Sigv4 support]({{site.url}}{{site.baseurl}}/benchmark/tutorials/sigv4/).

### PyPi

Expand All @@ -35,7 +35,7 @@ pip3 install opensearch-benchmark
```
{% include copy.html %}

After the installation completes, verify that OSB is running by entering the following command:
After the installation completes, verify that OpenSearch Benchmark is running by entering the following command:

```bash
opensearch-benchmark --help
Expand Down Expand Up @@ -94,47 +94,47 @@ docker run opensearchproject/opensearch-benchmark -h
{% include copy.html %}
## Running your first Benchmark
## Running your first benchmark
You can now run your first benchmark. For your first benchmark, you'll use the [geonames](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/geonames) workload.
### Understanding workload command flags
Benchmarks are run using the [`execute-test`]({{site.url}}{{site.baseurl}}/benchmark/commands/execute-test/) with the following command flags:
Benchmarks are run using the [`execute-test`]({{site.url}}{{site.baseurl}}/benchmark/commands/execute-test/) command with the following command flags:
For additional `execute_test` command flags, see the [execute-test]({{site.url}}{{site.baseurl}}/benchmark/commands/execute-test/) reference. Some commonly used options are `--workload-params`, `--exclude-tasks`, and `--include-tasks`
For additional `execute_test` command flags, see the [execute-test]({{site.url}}{{site.baseurl}}/benchmark/commands/execute-test/) reference. Some commonly used options are `--workload-params`, `--exclude-tasks`, and `--include-tasks`.
{: .tip}
* `--pipeline=benchmark-only` : Informs OSB that users wants to provide their own OpenSearch cluster.
- `workload=geonames`: The name of workload the OpenSearch Benchmark uses.
* `--target-host="<OpenSearch Cluster Endpoint>"`: Indicates the target cluster or host that OSB will benchmark. Enter the endpoint to your OpenSearch cluster here.
- `workload=geonames`: The name of workload used by OpenSearch Benchmark.
* `--target-host="<OpenSearch Cluster Endpoint>"`: Indicates the target cluster or host that will be benchmarked. Enter the endpoint of your OpenSearch cluster here.
* `--client-options="basic_auth_user:'<Basic Auth Username>',basic_auth_password:'<Basic Auth Password>'"`: The username and password for your OpenSearch cluster.
* `--test-mode`: Allows users to run the workload without running it for the entire duration. When this flag is present, OSB runs the first thousand operations of each task in the workload. This is only meant for sanity checks and the metrics produced are meaningless.
* `--test-mode`: Allows a user to run the workload without running it for the entire duration. When this flag is present, Benchmark runs the first thousand operations of each task in the workload. This is only meant for sanity checks---the metrics produced are meaningless.
The `--distribution-version`, which indicates which OpenSearch version OSB will use when provisioning. When run, the `execute-test` command will parse the correct distribution version when it connects to the OpenSearch cluster.
The `--distribution-version`, which indicates which OpenSearch version Benchmark will use when provisioning. When run, the `execute-test` command will parse the correct distribution version when it connects to the OpenSearch cluster.
### Running the workload
If you installed OSB with PyPi, customize and use the following command:
If you installed Benchmark with PyPi, customize and use the following command:
```bash
opensearch-benchmark execute-test --pipeline=benchmark-only --workload=geonames --target-host="<OpenSearch Cluster Endpoint>" --client-options="basic_auth_user:'<Basic Auth Username>',basic_auth_password:'<Basic Auth Password>'" --test-mode
```
{% include copy.html %}
If you installed OSB with Docker, customize and use the following command:
If you installed Benchmark with Docker, customize and use the following command:
```bash
docker run opensearchproject/opensearch-benchmark execute-test --pipeline=benchmark-only --workload=geonames --target-host="<OpenSearch Cluster Endpoint>" --client-options="basic_auth_user:'<Basic Auth Username>',basic_auth_password:'<Basic Auth Password>'" --test-mode
```
{% include copy.html %}
When the `execute_test` command runs, all tasks and operations inside the `geonames` workload run sequentially.
When the `execute_test` command runs, all tasks and operations in the `geonames` workload run sequentially.
### Understanding the results
OSB returns the following response once the Benchmark completes:
Benchmark returns the following response once the benchmark completes:
```bash
-----------------------------------------------------
Expand Down Expand Up @@ -385,17 +385,17 @@ OSB returns the following response once the Benchmark completes:
--------------------------------
```
Each task ran by the `geonames` workload represents a specific OpenSearch API operation, such as Bulk or Search, that was performed when the test was ran. Each task in the output summary contains the following information:
Each task run by the `geonames` workload represents a specific OpenSearch API operation---such as Bulk or Search---that was performed when the test was run. Each task in the output summary contains the following information:
* **Throughput:** The number of successful OpenSearch operations per second.
* **Latency:** The time, including wait time, it took for the request and the response to be sent out and received by OSB.
* **Service Time:** The time, excluding wait time, it took for the request and the response to be sent out and received by OSB.
* **Error Rate:** Percent of operations run during the task that were not successful or (200s).
* **Latency:** The amount of time, including wait time, taken for the request and the response to be sent and received by Benchmark.
* **Service Time:** The amount of time, excluding wait time, taken for the request and the response to be sent and received by Benchmark.
* **Error Rate:** The percentage of operations run during the task that were not successful or returned a 200 error code.
## Next steps
To learn more about OpenSearch Benchmark:
See the following resources to learn more about OpenSearch Benchmark:
- [User guide]({{site.url}}{{site.baseurl}}/benchmark/user-guide/index/): Dive deep into how OpenSearch Benchmark can you help you track the performance of your cluster.
- [Tutorials]({{site.url}}{{site.baseurl}}/benchmark/tutorials/index/): Use step-by-step guides for more advanced Benchmarking configurations and functionality.
12 changes: 6 additions & 6 deletions _benchmark/tutorials/sigv4.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
layout: default
title: SigV4 Support
title: AWS Signature Version 4 support
nav_order: 70
parent: Tutorials
---

# Running OpenSearch Benchmark with AWS Sigv4
# Running OpenSearch Benchmark with AWS Signature Version 4

OpenSearch Benchmark supports AWS Signature Version 4 (Sigv4) authentication. To run with Sigv4, use the following steps:
OpenSearch Benchmark supports AWS Signature Version 4 authentication. To run Benchmark with Signature Version 4, use the following steps:

1. Set up an [IAM user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html) and offer it access to your OpenSearch cluster that uses SigV4 Authentication.
1. Set up an [IAM user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html) and provide it access to the OpenSearch cluster using Signature Version 4 authentication.

2. Set up the following environment variables for your IAM user.
2. Set up the following environment variables for your IAM user:

```bash
OSB_AWS_ACCESS_KEY_ID=<<IAM USER AWS ACCESS KEY ID>
Expand All @@ -21,7 +21,7 @@ OpenSearch Benchmark supports AWS Signature Version 4 (Sigv4) authentication. To
```
{% include copy.html %}
3. Customize and run the following `execute-test` command with the ` --client-options=amazon_aws_log_in:environment` flag. This flag tells OpenSearch Benchmark where you've exported your credentials.
3. Customize and run the following `execute-test` command with the ` --client-options=amazon_aws_log_in:environment` flag. This flag tells OpenSearch Benchmark the location of your exported credentials.
```bash
opensearch-benchmark execute-test \
Expand Down
18 changes: 9 additions & 9 deletions _benchmark/user-guide/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ Before using OpenSearch Benchmark, familiarize yourself with the following conce

## Core concepts and definitions

- **Workload**: The description of one or more benchmarking scenarios that use a specific document corpus from which to perform a benchmark against your cluster. The document corpus contains any indexes, data files, and operations invoked when the workflow runs. You can list the available workloads by using `opensearch-benchmark list workloads` or view any included workloads inside the [OpenSearch Benchmark Workloads repository](https://github.com/opensearch-project/opensearch-benchmark-workloads/). For more details about the parts that compose a workload, see [Anatomy of a workload](#anatomy-of-a-workload) For information about building a custom workload, see [Creating custom workloads]({{site.url}}{{site.baseurl}}/benchmark/creating-custom-workloads/).
- **Workload**: The description of one or more benchmarking scenarios that use a specific document corpus to perform a benchmark against your cluster. The document corpus contains any indexes, data files, and operations invoked when the workflow runs. You can list the available workloads by using `opensearch-benchmark list workloads` or view any included workloads in the [OpenSearch Benchmark Workloads repository](https://github.com/opensearch-project/opensearch-benchmark-workloads/). For more information about the elements of a workload, see [Anatomy of a workload](#anatomy-of-a-workload). For information about building a custom workload, see [Creating custom workloads]({{site.url}}{{site.baseurl}}/benchmark/creating-custom-workloads/).

- **Pipeline**: A series of steps before and after a workload is run that determines benchmark results. OpenSearch Benchmark supports three pipelines:
- **Pipeline**: A series of steps occurring before and after a workload is run that determines benchmark results. OpenSearch Benchmark supports three pipelines:
- `from-sources`: Builds and provisions OpenSearch, runs a benchmark, and then publishes the results.
- `from-distribution`: Downloads an OpenSearch distribution, provisions it, runs a benchmark, and then publishes the results.
- `benchmark-only`: The default pipeline. Assumes an already running OpenSearch instance, runs a benchmark on that instance, and then publishes the results.
Expand All @@ -22,12 +22,12 @@ Before using OpenSearch Benchmark, familiarize yourself with the following conce

A workload is a specification of one or more benchmarking scenarios. A workload typically includes the following:

- One or more data streams that are ingested into indices
- One or more data streams that are ingested into indexes
- A set of queries and operations that are invoked as part of the benchmark

## Anatomy of a workload

The following example workload shows all of the essential elements needed to create a workload.json file. You can run this workload in your own benchmark configuration to understand how all of the elements work together:
The following example workload shows all of the essential elements needed to create a `workload.json` file. You can run this workload in your own benchmark configuration to understand how all of the elements work together:

```json
{
Expand Down Expand Up @@ -92,27 +92,27 @@ The following example workload shows all of the essential elements needed to cre

A workload usually includes the following elements:

- [indices]({{site.url}}{{site.baseurl}}/benchmark/workloads/indices/): Defines the relevant indices and index templates used for the workload.
- [indices]({{site.url}}{{site.baseurl}}/benchmark/workloads/indices/): Defines the relevant indexes and index templates used for the workload.
- [corpora]({{site.url}}{{site.baseurl}}/benchmark/workloads/corpora/): Defines all document corpora used for the workload.
- `schedule`: Defines operations and the order in which the operations run in-line. Alternatively, you can use `operations` to group operations and the `test_procedures` parameter to specify the order of operations.
- `schedule`: Defines operations and the order in which the operations run inline. Alternatively, you can use `operations` to group operations and the `test_procedures` parameter to specify the order of operations.
- `operations`: **Optional**. Describes which operations are available for the workload and how they are parameterized.

### Indices

To create an index, specify its `name`. To add definitions to your index, use the `body` option and point it to the JSON file containing the index definitions. For more information, see [indices]({{site.url}}{{site.baseurl}}/benchmark/workloads/indices/). For more information, see [indices]({{site.url}}{{site.baseurl}}/benchmark/workloads/indices/).
To create an index, specify its `name`. To add definitions to your index, use the `body` option and point it to the JSON file containing the index definitions. For more information, see [indices]({{site.url}}{{site.baseurl}}/benchmark/workloads/Indices/).

### Corpora

The `corpora` element requires the name of the index containing the document corpus, for example, `movies`, and a list of parameters that define the document corpora. This list includes the following parameters:

- `source-file`: The file name that contains the workload's corresponding documents. When using OpenSearch Benchmark locally, documents are contained in a JSON file. When providing a `base_url`, use a compressed file format: `.zip`, `.bz2`, `.gz`, `.tar`, `.tar.gz`, `.tgz`, or `.tar.bz2`. The compressed file must have one JSON file containing the name.
- `document-count`: The number of documents in the `source-file`, which determines which client indices correlate to which parts of the document corpus. Each N client receives an Nth of the document corpus. When using a source that contains a document with a parent-child relationship, specify the number of parent documents.
- `document-count`: The number of documents in the `source-file`, which determines which client indexes correlate to which parts of the document corpus. Each N client receives an Nth of the document corpus. When using a source that contains a document with a parent-child relationship, specify the number of parent documents.
- `uncompressed-bytes`: The size, in bytes, of the source file after decompression, indicating how much disk space the decompressed source file needs.
- `compressed-bytes`: The size, in bytes, of the source file before decompression. This can help you assess the amount of time needed for the cluster to ingest documents.

### Operations

The `operations` element lists the OpenSearch API operations performed by the workload. For example, you can set an operation to `create-index`, which creates an index in the test cluster that OpenSearch Benchmark can write documents into. Operations are usually listed inside of `schedule`.
The `operations` element lists the OpenSearch API operations performed by the workload. For example, you can set an operation to `create-index`, an index in the test cluster to which OpenSearch Benchmark can write documents. Operations are usually listed inside of `schedule`.

### Schedule

Expand Down
4 changes: 2 additions & 2 deletions _benchmark/user-guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ nav_order: 5
has_children: true
---

# OpenSearch Benchmark user guide
# OpenSearch Benchmark User Guide

The OpenSearch Benchmark user guide includes information core [concepts]({{site.url}}{{site.baseurl}}/benchmark/user-guide/concepts/), [installation]({{site.url}}{{site.baseurl}}/benchmark/installing-benchmark/), and [configuration options]({{site.url}}{{site.baseurl}}/benchmark/configuring-benchmark/) to help you get the most of OpenSearch Benchmark.
The OpenSearch Benchmark User Guide includes core [concepts]({{site.url}}{{site.baseurl}}/benchmark/user-guide/concepts/), [installation]({{site.url}}{{site.baseurl}}/benchmark/installing-benchmark/) instructions, and [configuration options]({{site.url}}{{site.baseurl}}/benchmark/configuring-benchmark/) to help you get the most out of OpenSearch Benchmark.
Loading

0 comments on commit 9535205

Please sign in to comment.