Skip to content

Commit

Permalink
feat(OTel): Add more formatting
Browse files Browse the repository at this point in the history
  • Loading branch information
rhetoric101 committed May 29, 2024
1 parent 67c7021 commit 3139dba
Showing 1 changed file with 36 additions and 31 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,19 @@ tags:
- helm
metaDescription: You can collect Kafka metrics from Confluent using the OpenTelemetry Collector on Kubernetes.
freshnessValidatedDate: 2024-05-28
redirects:
- /docs/more-integrations/open-source-telemetry-integrations/opentelemetry/collector/collector-configuration-examples/opentelemetry-collector-kafka-confluentcloud/
---

You can collect metrics about your Confluent Cloud-managed Kafka deployment with the OpenTelemetry Collector. The collector is a component of OpenTelemetry that collects, processes, and exports telemetry data to New Relic (or any observability backend).

This integration works by running a Prometheus receiver configuration inside the OpenTelemetry collector, which scrapes [Confluent Cloud's metrics API](https://api.telemetry.confluent.cloud/docs/descriptors/datasets/cloud?_ga=2.183807142.1264186867.1705940186-6520871.1686857317&_gl=1*1te8jue*_ga*NjUyMDg3MS4xNjg2ODU3MzE3*_ga_D2D3EGKSGD*MTcwNjAzNzYwOS41Ni4wLjE3MDYwMzc2MDkuNjAuMC4w) and exports that data to New Relic.

## Set up monitoring [#set-up-monitoring]

Complete the steps below to collect Kafka metrics from Confluent and export them to New Relic.

<Steps>
<Step>
## Make sure you're set up
### Make sure you're ready to get started [#get-started]

Before you start, you need to have the <InlinePopover type="licenseKey" /> for the account you want to report data to. You should also verify that:

Expand All @@ -33,46 +33,51 @@ Complete the steps below to collect Kafka metrics from Confluent and export them

</Step>
<Step>
## Download or clone the OpenTelemetry tech partner examples repo
### Download or clone the OpenTelemetry tech partner examples repository [#download-repo]

Download [New Relic's OpenTelemetry tech partnership examples repository](https://github.com/newrelic-experimental/tech-partner-opentelemetry-integrations) as this setup uses its example collector configuration. Once installed, open the [Confluent Cloud example](https://github.com/newrelic-experimental/tech-partner-opentelemetry-integrations/confluent-cloud) directory. For more information, you can check the `README` there as well.
Download [New Relic's OpenTelemetry tech partnership examples repository](https://github.com/newrelic-experimental/tech-partner-opentelemetry-integrations) as this setup uses its example collector configuration. Once installed, open the [Confluent Cloud example](https://github.com/newrelic-experimental/tech-partner-opentelemetry-integrations/confluent-cloud) directory. For more information, you can check the `README` there as well.

</Step>
<Step>
## Run the example [#run-the-example]
### Run the example [#run-the-example]

Complete either the Helm or Kubernetes instructions below, depending on your environment.

### Helm example [#run-helm-example]
Complete either the Helm or Kubernetes instructions below, depending on your environment.

#### Helm example [#run-helm-example]

1. From the `./confluent-cloud/helm` directory, update the variables in the `values.yaml` file. For more information on the individual variables, see the [local variables](#local-variables) table below.

2. Run the collector by using the following command:

```bash
# Open the confluent cloud helm example directory
cd ./confluent-cloud/helm

# Make sure to set environment variables.

# Install and run helm
helm install ./helm
```

From the `./confluent-cloud/helm` directory, update the variables in the `values.yaml` accordingly. For more information on the individual variables, reference the [Local variables](#local-variable-table) table below.

Next, run the collector by using the following command
```bash
# Open the confluent cloud helm example directory
cd ./confluent-cloud/helm
#### Kubernetes example [#run-kubernetes-example]

# Make sure to set environment variables.
1. From the `./confluent-cloud/k8s` directory, update the variables in the in the `./k8s/deployment.yaml` file. For more information on the individual variables, see the [local variables](#local-variables) table below.

# Install and run helm
helm install ./helm
```
### Run the Kubernetes example [#run-kubernetes-example]
2. Once the variables are set, run the following command to apply the collector to your Kubernetes cluster.

From the `./confluent-cloud/k8s` directory, update the variables in the in the `./k8s/deployment.yaml` file. For more information on the individual variables, reference the [Local variables](#local-variables-table) table below.
```bash
# Open the Confluent Cloud k8s example directory
cd ./confluent-cloud/k8s

Once the variables are set, run the following command to apply the collector to your Kubernetes cluster.
# Make sure to set environment variables

```bash
# Open the Confluent Cloud k8s example directory
cd ./confluent-cloud/k8s
# Apply the collector to the cluster
kubectl apply -f ./k8s
```

# Make sure to set environment variables

# Apply the collector to the cluster
kubectl apply -f ./k8s
```
#### Local variables [#local-variable-table]
#### Local variables for Helm and Kubernetes [#local-variables]

<table>
<thead>
Expand Down Expand Up @@ -172,7 +177,7 @@ Complete the steps below to collect Kafka metrics from Confluent and export them
</Step>
<Step>

## View your data in New Relic
### View your data in New Relic [#view-your-data]

You can view your Confluent Cloud data in a few different ways.

Expand All @@ -183,7 +188,7 @@ Complete the steps below to collect Kafka metrics from Confluent and export them
</Step>
</Steps>

### Confluent Cloud metrics [#confluent-metrics]
## Confluent Cloud metrics [#confluent-metrics]

This integration covers all the *Exportable* metrics within the [Confluent Cloud Metrics API](https://api.telemetry.confluent.cloud/docs/descriptors/datasets/cloud). We have a partial list of the *Exportable* metrics below:

Expand Down

0 comments on commit 3139dba

Please sign in to comment.