diff --git a/Makefile b/Makefile index dd85f3b..a92b7fa 100644 --- a/Makefile +++ b/Makefile @@ -64,9 +64,14 @@ eck: cert-manager otel-operator eck-operator dora: default kubectl apply -k ./collectors/webhook/ +.PHONY: gha-traces +gha-traces: default + kubectl apply -k ./collectors/githubactionevents/ + .PHONY: ngrok ngrok: helm repo add ngrok https://ngrok.github.io/kubernetes-ingress-controller + helm repo update kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml helm upgrade -i ngrok-ingress-controller ngrok/kubernetes-ingress-controller \ --namespace ngrok-ingress \ diff --git a/README.md b/README.md index c1e2949..07925d0 100644 --- a/README.md +++ b/README.md @@ -6,14 +6,14 @@ This set of manifests gets a local obersvability stack up and running quickly. It installs the following services into your local kubernetes cluster: -* Grafana -* Prometheus -* Tempo -* Loki -* Certificate Manager -* OpenTelemetry Controller -* Liatrio OpenTelemetry Collector -* NGrok Ingress and API Gateway Controller +- Grafana +- Prometheus +- Tempo +- Loki +- Certificate Manager +- OpenTelemetry Controller +- Liatrio OpenTelemetry Collector +- NGrok Ingress and API Gateway Controller ## Prerequisites @@ -75,6 +75,7 @@ presumes that you have a free NGrok account, an API Key, and an AuthToken. ``` 5. Run `make ngrok` to setup the controller. + > You may need to delete the ngrok controller pod if it's not creating the route. TODO: move to tailscale 6. Update the [webhook route config](./collectors/webhook/ngrok-route.yaml) with your permanent domain in the host rules (see example below): @@ -82,21 +83,62 @@ presumes that you have a free NGrok account, an API Key, and an AuthToken. spec: ingressClassName: ngrok rules: - # Change this to match your NGrok permanent domain - - host: example.ngrok-free.app + # Change this to match your NGrok permanent domain + - host: example.ngrok-free.app ``` 7. Run `make dora` +## GitHub Actions Event Tracing + +There is currently a [New Component][component] that has been submitted for +discussion within the OpenTelemetry community. This component builds traces out +of GitHub Action workflow events. This can be a useful piece of the puzzle for +understanding how your CI/CD pipeline is functioning, by leveraging context +rich traces. This section allows you to run this component locally for testing +purposes. It is HIGHLY EXPERIMENTAL and should not be used in production. +Originating documentation [can be found here][gha-docs]. + +[component]: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27460 +[gha-docs]: https://github.com/krzko/opentelemetry-collector-contrib/tree/feat-add-githubactionseventreceiver/receiver/githubactionsreceiver + +> Much is the same as the DORA configuration because the component was originally based off the webhook event receiver. You will need to ensure that your repository emits workflow events. + +1. From the [NGrok dashboard][ngrok-dash] get your [API Key][ngrok-api] from NGrok. +2. Get your [Auth Token][ngrok-api] from NGrok. +3. Get your [free permanent domain][ngrok-domain] from NGrok. +4. Export your env vars: + + ```bash + export NGROK_AUTHTOKEN=authtoken + export NGROK_API_KEY=apikey + ``` + +5. Run `make ngrok` to setup the controller. + > You may need to delete the ngrok controller pod if it's not creating the route. TODO: move to tailscale +6. Update the [github action event route config](./collectors/githubactionevents/ngrok-route.yaml) + with your permanent domain in the host rules (see example below): + + ```yaml + spec: + ingressClassName: ngrok + rules: + # Change this to match your NGrok permanent domain + - host: example.ngrok-free.app + ``` + +7. Run `make gha-traces` +8. Port forward Jaeger & view the traces. + ## Tracing Demo ![Logo](content/logo3.png) 1. To run the demo, you will need to have a Kubernetes cluster running locally - as well as `kubectl` installed. We will use [k3d](https://k3d.io/) to create - a local cluster. If you do not have these installed, you can install them by + as well as `kubectl` installed. We will use [k3d](https://k3d.io/) to create + a local cluster. If you do not have these installed, you can install them by running one of the followings commands depending on your OS: **Linux** @@ -127,7 +169,7 @@ presumes that you have a free NGrok account, an API Key, and an AuthToken. make apply-traces ``` -4. Verify that the namespaces are present and the pods are running. They should +4. Verify that the namespaces are present and the pods are running. They should look like this: ![kubectl get namespaces](content/namespaces.png) @@ -141,7 +183,7 @@ presumes that you have a free NGrok account, an API Key, and an AuthToken. visiting `http://localhost:3000` in your browser. The dashboard will be the only one in the demo folder and will look like this: ![Grafana Dashboard](content/dashboard.png) - + > [!IMPORTANT] Grafana will ask for a login which will just be the default > credentials of `username:admin password:admin`. It will ask you to change > it but you can skip this step if you would like. @@ -155,11 +197,11 @@ make delete-traces ## Tracing We have an instrumented version of the flux-iac Tofu Controller which is part of -what makes this demo possible. Our fork with the changes are +what makes this demo possible. Our fork with the changes are [here][tofu-controller] The other core piece of the demo is our instrumented version of the OpenTofu -binary. Similarly our fork with the changes are +binary. Similarly our fork with the changes are [here][open-tofu] ## Configuration @@ -172,7 +214,7 @@ configuration with your own terraform, you will need to do the following. ![Source](content/source.png) 2. Update one of the `terraform.yml` files in the same folder so it references the name of the object you - created with the `source_control.yml` file in the `sourceRef` field. Then + created with the `source_control.yml` file in the `sourceRef` field. Then update the `path` field with the specific path to the terraform resources you want to use inside the repository. ![Source](content/terraform.png) @@ -182,14 +224,14 @@ configuration with your own terraform, you will need to do the following. them to be deployed with the rest of the resources 4. Run `make apply-traces` to update the resources in the cluster with the new -configuration. - -> * For the purposes of the tracing demo these will by default be configured -> to apply null resources to the cluster since deploying resources to a -> cloud provider requires an additional auth setup that is not done here. -> * Deploying kubernetes resources is also possible but requires you to update -> the `tf-runner` service account with a cluster role that has permissions to -> act on those resources. + configuration. + +> - For the purposes of the tracing demo these will by default be configured +> to apply null resources to the cluster since deploying resources to a +> cloud provider requires an additional auth setup that is not done here. +> - Deploying kubernetes resources is also possible but requires you to update +> the `tf-runner` service account with a cluster role that has permissions to +> act on those resources. [brew]: https://brew.sh/ [dd]: https://www.docker.com/products/docker-desktop/ diff --git a/collectors/githubactionevents/colconfig.yaml b/collectors/githubactionevents/colconfig.yaml new file mode 100755 index 0000000..7ac0b32 --- /dev/null +++ b/collectors/githubactionevents/colconfig.yaml @@ -0,0 +1,40 @@ +--- +extensions: + health_check: + +receivers: + # This is a test component, originally opened in contrib: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/27460 + # For docs see this readme https://github.com/krzko/opentelemetry-collector-contrib/tree/feat-add-githubactionseventreceiver/receiver/githubactionsreceiver + githubactions: + endpoint: 0.0.0.0:8088 + path: /events + # secret: "" + +processors: + batch: + send_batch_size: 100 + timeout: 10s + +exporters: + debug: + verbosity: detailed + sampling_initial: 2 + sampling_thereafter: 500 + + otlp: + endpoint: http://gateway-collector.collector.svc.cluster.local:4317 + tls: + insecure: true + +service: + extensions: + - health_check + + pipelines: + traces: + receivers: + - githubactions + processors: [batch] + exporters: + - debug + - otlp diff --git a/collectors/githubactionevents/kustomization.yaml b/collectors/githubactionevents/kustomization.yaml new file mode 100644 index 0000000..aeb1312 --- /dev/null +++ b/collectors/githubactionevents/kustomization.yaml @@ -0,0 +1,54 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +namespace: collector + +resources: + - ../base/ + - sa.yaml + - ngrok-route.yaml + +patches: + - target: + kind: OpenTelemetryCollector + name: collector + patch: |- + - op: replace + path: /spec/image + value: ghcr.io/krzko/otelcol-distributions/githubactions:0.99.1 + - op: replace + path: /metadata/name + value: otel-gha-collector + - op: replace + path: /spec/serviceAccount + value: otel-gha-collector + - op: replace + path: /spec/ports + value: + - name: gha-port + port: 8088 + protocol: TCP + targetPort: 8088 + +configMapGenerator: + - name: collector-config + files: + - colconfig.yaml + options: + annotations: + config.kubernetes.io/local-config: "true" + +replacements: + - source: + kind: ConfigMap + name: collector-config + fieldPath: data.[colconfig.yaml] + + targets: + - select: + kind: OpenTelemetryCollector + fieldPaths: + - spec.config + options: + create: true diff --git a/collectors/githubactionevents/ngrok-route.yaml b/collectors/githubactionevents/ngrok-route.yaml new file mode 100644 index 0000000..4680703 --- /dev/null +++ b/collectors/githubactionevents/ngrok-route.yaml @@ -0,0 +1,19 @@ +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: otel-gha-ingress + # annotations: + # k8s.ngrok.com/modules: ngrok-module-set +spec: + ingressClassName: ngrok + rules: + - host: giraffe-suited-only.ngrok-free.app + http: + paths: + - path: /events + pathType: Prefix + backend: + service: + name: otel-gha-collector-collector + port: + number: 8088 diff --git a/collectors/githubactionevents/sa.yaml b/collectors/githubactionevents/sa.yaml new file mode 100644 index 0000000..c768832 --- /dev/null +++ b/collectors/githubactionevents/sa.yaml @@ -0,0 +1,12 @@ +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + labels: + app.kubernetes.io/component: opentelemetry-collector + app.kubernetes.io/instance: otel-gha.otel-gha-collector + app.kubernetes.io/managed-by: opentelemetry-operator + app.kubernetes.io/name: otel-gha-collector + app.kubernetes.io/part-of: opentelemetry + name: otel-gha-collector + namespace: collector