Skip to content

Commit

Permalink
docs: update testkube integration (#3833)
Browse files Browse the repository at this point in the history
  • Loading branch information
jorgeepc authored May 1, 2024
1 parent 09fb312 commit eaa473d
Show file tree
Hide file tree
Showing 3 changed files with 100 additions and 39 deletions.
Binary file added docs/docs/tools-and-integrations/img/testkube.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
139 changes: 100 additions & 39 deletions docs/docs/tools-and-integrations/testkube.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,29 +8,40 @@ keywords:
- observability
- distributed tracing
- testing
- testkube
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
---

[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions.
[Tracetest](https://app.tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that allows you to test your distributed application. It uses data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior described by your test definitions.

[Testkube](https://testkube.io/) is a Kubernetes-native testing framework for Testers and Developers that allows you to automate the executions of your existing testing tools inside your Kubernetes cluster, removing all the complexity from your CI/CD/GitOps pipelines.

For Testkube, tests are meant to be part of a cluster's state and can be executed as needed:
## Why is this important?

- Manually via kubectl CLI.
- Externally triggered via API (CI, external tooling, etc).
- Automatically on deployment of annotated/labeled services/pods/etc.
Integrating Tracetest with Testkube enhances testing processes by leveraging Kubernetes-native capabilities, improving scalability, and reliability of testing workflows for distributed applications.

By using the [Testkube Tracetest Executor](https://github.com/kubeshop/testkube-executor-tracetest) you can unlock Testkube's capacity in conjunction with Tracetest, and leverage the work you have already done to instrument your services.
<div style={{ position: "relative", paddingBottom: "62.5%", height: "0" }}>
<iframe
src="https://www.youtube.com/embed/nAp3zYgykok"
title="Trace-based testing in Kubernetes with Testkube and Tracetest"
frameborder="0"
webkitallowfullscreen
mozallowfullscreen
allowfullscreen
width="100%"
style={{ position: "absolute", top: "0", left: "0", width: "100%", height: "100%" }}
></iframe>
</div>

:::info
If you are using the latest version of Testkube, the Tracetest Executor will be bundled in Testkube by default. Read more in the Testkube docs, [here](https://docs.testkube.io/test-types/executor-tracetest/).
## The Testkube Tracetest Executor

Or, check out the hands-on workshop on YouTube!
For Testkube, tests are meant to be part of a cluster's state and can be executed as needed:

<iframe width="100%" height="250" src="https://www.youtube.com/embed/nAp3zYgykok" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
- Manually via kubectl CLI.
- Externally triggered via API (CI, external tooling, etc).
- Automatically on deployment of annotated/labeled services/pods/etc.

:::
By using the [Testkube Tracetest Executor](https://docs.testkube.io/test-types/executor-tracetest) you can unlock Testkube's capacity in conjunction with Tracetest, and leverage the work you have already done to instrument your services.

## How does it work?

Expand All @@ -43,14 +54,16 @@ sequenceDiagram
executor CRDs-->>-testkube: Send details
testkube->>+tracetest executor job: Schedules execution
tracetest executor job->>+tracetest executor job: Configure Tracetest CLI
tracetest executor job->>+tracetest server: Executes the Tracetest test run
tracetest server->>+instrumented service: Trigger request
instrumented service-->>-tracetest server: Get response
tracetest executor job->>+tracetest: Executes the Tracetest test run
tracetest->>+tracetest agent: Publish test run job
tracetest agent->>+instrumented service: Trigger request
instrumented service-->>-tracetest agent: Get response
instrumented service->>+data store: Send telemetry data
tracetest server->>+data store: Fetch trace
data store-->>-tracetest server: Get trace
tracetest server->>+tracetest server: Run assertions
tracetest server-->>-tracetest executor job: Return test run results
tracetest agent->>+data store: Fetch trace
data store-->>-tracetest agent: Get trace
tracetest agent-->>-tracetest: Sync and store response and trace
tracetest->>+tracetest: Run analyzer and assertions
tracetest-->>-tracetest executor job: Return test run results
tracetest executor job-->>-testkube: Return test run results
testkube-->>-testkube client: Send details
```
Expand All @@ -61,34 +74,45 @@ This guide will show how to use Testkube alongside Tracetest to run your tests i

### Prerequisites

In your Kubernetes cluster you should have:
**Tracetest account**:

- Sign up to [`app.tracetest.io`](https://app.tracetest.io) or follow the [get started](/getting-started/installation) docs.
- Create an [organization](/concepts/organizations) and [environment](/concepts/environments).
- Deploy the [Tracetest Agent](/configuration/agent) in your cluster.
- Create an [environment token](/concepts/environment-tokens).

**Testkube account**:

1. `Testkube`: Use HELM or the Testkube CLI to [install](https://kubeshop.github.io/testkube/installing) Testkube Server components in your cluster.
2. `Tracetest`: You need a running instance of [Tracetest](https://docs.tracetest.io/getting-started/installation/) or [Tracetest Core](https://docs.tracetest.io/core/getting-started/installation/) which is going to be executing your assertions. To do so, you can follow the instructions defined in the Tracetest [documentation](https://docs.tracetest.io/getting-started/overview).
3. `OpenTelemetry Instrumented Service`: In order to generate traces and spans, the service under test must support the basics for [propagation](https://opentelemetry.io/docs/reference/specification/context/api-propagators/) through HTTP requests, and also store traces and spans into a Data Store Backend (Jaeger, Grafana Tempo, OpenSearch, etc) or use the [OpenTelemetry Collector](https://docs.tracetest.io/configuration/overview#using-tracetest-without-a-trace-data-store).
- Sign up to [`app.testkube.io`](https://app.testkube.io) or follow the [get started](https://docs.testkube.io/articles/getting-started) docs.
- Create a Testkube environment.
- Deploy the [Testkube Agent](https://docs.testkube.io/testkube-pro/articles/installing-agent) in your cluster.

Remember, in your Kubernetes cluster you should have:

1. `Tracetest Agent`.
2. `Testkube Agent`.
3. `OpenTelemetry Instrumented Service`: In order to generate traces and spans, the service under test must support the basics for [propagation](https://opentelemetry.io/docs/reference/specification/context/api-propagators/) through HTTP requests, and also store traces and spans into a Tracing Backend (Jaeger, Grafana Tempo, OpenSearch, etc) or use the [OpenTelemetry Collector](https://docs.tracetest.io/configuration/overview#using-tracetest-without-a-trace-data-store). If you are using a Tracing Backend, the Tracetest Agent requires network access to it. In case of using the OpenTelemetry Collector, you need to setup your collector to send trace data to the Tracetest Agent.

On your machine you should have:

1. `Kubectl` [installed](https://kubernetes.io/docs/tasks/tools/).
2. `Testkube CLI` [installed](https://kubeshop.github.io/testkube/installing#1-installing-the-testkube-cli).
2. `Testkube CLI` [installed](https://docs.testkube.io/articles/install/cli).

With everything set up, we will start configuring Testkube and Tracetest.

### 1. Deploy the Tracetest Executor
### 1. Connect your Testkube CLI to your Testkube Environment

Testkube works with the concept of [Executors](https://kubeshop.github.io/testkube/test-types/executor-custom). An Executor is a wrapper around a testing framework (Tracetest in this case) in the form of a Docker container and runs as a Kubernetes job. To start, you need to register and deploy the Tracetest executor in your cluster using the Testkube CLI:
To use the Testkube CLI with your Testkube account you need to set the CLI Context. For that you need a Testkube Pro token. When the token is created, you are ready to change the Testkube CLI context:

```bash
kubectl testkube create executor --image kubeshop/testkube-executor-tracetest:latest --types "tracetest/test" --name tracetest-executor --icon-uri icon --content-type string --content-type file-uri
testkube set context -c cloud -e testkube-environment-id -o testkube-organization-id -k testkube-token
```

:::info
If you are using the latest version of Testkube, the Tracetest Executor will be bundled in Testkube by default. Read more in the Testkube docs, [here](https://docs.testkube.io/test-types/executor-tracetest/).
:::
For more information see the Testkube [Connecting from the CLI](https://docs.testkube.io/testkube-pro/articles/managing-cli-context) docs.

### 2. Create a Test

Now you need a Tracetest test. Have a look at the [Tracetest documentation](https://docs.tracetest.io/cli/creating-tests) for details on writing tests. Here is a simple test definition example:
For this step you need a Tracetest test. Have a look at the [Tracetest documentation](/cli/creating-tests) for details on writing tests. Here is a simple test definition example:

```yaml
type: Test
Expand All @@ -113,25 +137,62 @@ spec:
- attr:db.name = "pokeshop"
```
Execute the following command to create the test executor object in Testkube. Do not forget to provide the path to your Tracetest definition file using the `--file` argument, and also the follow variables:
- **If you are using [Tracetest Core](https://docs.tracetest.io/core/getting-started/installation/)**: Define the Tracetest Server endpoint using the `TRACETEST_ENDPOINT` `--variable` (remember that the TRACETEST_ENDPOINT should be reachable from the Testkube instance).
Execute the following command to create the test executor object in Testkube. Do not forget to provide the path to your Tracetest definition file using the `--file` argument, and also the following variables:

- `TRACETEST_TOKEN`: your [environment token](../concepts/environment-token.mdx).
- `TRACETEST_ENVIRONMENT`: your [environment id](../concepts/environments.mdx).
- `TRACETEST_ORGANIZATION`: your [organization id](../concepts/organizations.mdx).

```bash
kubectl testkube create test --file my/file/location.yaml --type "tracetest/test" --name pokeshop-tracetest-test --variable TRACETEST_ENDPOINT=http://tracetest
kubectl testkube create test --file my/file/location.yaml --type "tracetest/test" --name pokeshop-tracetest-test --variable TRACETEST_TOKEN=tracetest-token --variable TRACETEST_ENVIRONMENT=tracetest-environment-id --variable TRACETEST_ORGANIZATION=tracetest-organization-id
```

Note: In case you are doing a port forward to your Tracetest instance and you want to have the correct Tracetest URL in your results printed by the Testkube output, you can also provide an optional `TRACETEST_OUTPUT_ENDPOINT` variable (e.g. `--variable TRACETEST_OUTPUT_ENDPOINT=http://localhost:11633`).
### 3. Run the Test

- **If you are using [Tracetest](https://docs.tracetest.io/getting-started/installation/) on `app.tracetest.io`**: Define your [token](../concepts/environment-token.mdx), [environment](../concepts/environments.mdx) and [organization](../concepts/organizations.mdx) using the `TRACETEST_TOKEN`, `TRACETEST_ENVIRONMENT` and `TRACETEST_ORGANIZATION` variables
To see the integration working, run the test by executing the following command:

```bash
kubectl testkube create test --file my/file/location.yaml --type "tracetest/test" --name pokeshop-tracetest-test --variable TRACETEST_TOKEN=your-token --variable TRACETEST_ENVIRONMENT=your-environment-id --variable TRACETEST_ORGANIZATION=your-organization-id
kubectl testkube run test --watch pokeshop-tracetest-test
```

### 3. Run the Test
### 4. Explore the results

Finally, to see the integration working, run the test by executing the following command:
Finally, you can see the Tracetest results in your CLI:

```bash
kubectl testkube run test --watch pokeshop-tracetest-test
🚚 [TracetestRunner]: Preparing test run
🌍 Configuring Tracetest CLI with Token
🌍 Using arguments to configure CLI: [configure --token tracetest-token --organization tracetest-organization-id --environment tracetest-environment-id]
🚀 Configure command tracetest configure --token tracetest-token --organization tracetest-organization-id --environment tracetest-environment-id
🔬 Executing in directory :
$ tracetest configure --token tracetest-token --organization tracetest-organization-id --environment tracetest-environment-id
SUCCESS Successfully configured Tracetest CLI
✅ Execution succeeded
🌍 Using arguments to run test: [run test --file /data/test-content --output pretty]
🚀 Test run command tracetest run test --file /data/test-content --output pretty
🔬 Executing in directory :
$ tracetest run test --file /data/test-content --output pretty
✔ RunGroup: #58WmDuBSR (https://app.tracetest.io/organizations/tracetest-organization-id/environments/tracetest-environment-id/run/58WmDuBSR)
Summary: 1 passed, 0 failed, 0 pending
✔ Pokeshop - List (https://app.tracetest.io/organizations/tracetest-organization-id/environments/tracetest-environment-id/test/R5NITR14g/run/1/test) - trace id: 3cdcb56d6c226f7083f45d6b3d278051
✔ span[tracetest.span.type="http"]
✔ span[tracetest.span.type="database"]
```

You can click the run link to open the test run in the Tracetest app. From there you can explore the generated trace, update the assertions and get the test definition to execute your test once more.

![tracetest](./img/tracetest.gif)

You can also explore the test execution from the Testkube dashboard.

![testkube](./img/testkube.gif)

## What's Next?

After running this quickstart, you can now add Tracetest to the native CI/CD pipeline in your Kubernetes cluster. It allows you to execute scheduled test runs and synthetic tests. All while following the trace-based testing principle and enabling full in-depth assertions against trace data, not just the response.

## Learn More

Please visit our [documentation](https://docs.tracetest.io/) and join our [Slack Community](https://dub.sh/tracetest-community) for more info!

0 comments on commit eaa473d

Please sign in to comment.