From 65fbf75c726d5b2f6ffddb6020ca6efdc9e3995e Mon Sep 17 00:00:00 2001 From: Kevin Carter Date: Tue, 17 Dec 2024 10:29:20 -0600 Subject: [PATCH 1/3] fix: add fluentbit docs This change updates the docs to fix the missing fluentbit information from our install guide. While we had it doc'd we didn't have it published so this change removes the README file and pushes the content into the published set. Signed-off-by: Kevin Carter --- base-helm-configs/fluentbit/README.md | 7 ------- docs/infrastructure-fluentd.md | 18 ++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 19 insertions(+), 7 deletions(-) delete mode 100644 base-helm-configs/fluentbit/README.md create mode 100644 docs/infrastructure-fluentd.md diff --git a/base-helm-configs/fluentbit/README.md b/base-helm-configs/fluentbit/README.md deleted file mode 100644 index 8ff48dc7..00000000 --- a/base-helm-configs/fluentbit/README.md +++ /dev/null @@ -1,7 +0,0 @@ -# Install the fluentbit helm chart - -``` -helm repo add fluent https://fluent.github.io/helm-charts -helm repo update` -helm install --values fluentbit-helm-overrides.yaml fluentbit fluent/fluent-bit -``` diff --git a/docs/infrastructure-fluentd.md b/docs/infrastructure-fluentd.md new file mode 100644 index 00000000..43b70ab1 --- /dev/null +++ b/docs/infrastructure-fluentd.md @@ -0,0 +1,18 @@ +# Deploy Fluentbit + +This guide will help you deploy fluentbit to your kubernetes cluster. Fluentbit is a lightweight log shipper that can be used to send logs to loki. + +## Install the fluentbit helm chart + +``` shell +helm repo add fluent https://fluent.github.io/helm-charts +helm repo update +``` + +## Install the helm chart + +You will need to make changes depending on how you want to configure loki. Example files are included in this directory choose one relevant to your deploy + +``` shell +helm install --values fluentbit-helm-overrides.yaml fluentbit fluent/fluent-bit +``` diff --git a/mkdocs.yml b/mkdocs.yml index 0bcdca03..ef2739eb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -182,6 +182,7 @@ nav: - Memcached: infrastructure-memcached.md - Libvirt: infrastructure-libvirt.md - OVN: infrastructure-ovn-setup.md + - FluentBit: infrastructure-fluentbit.md - Loki: infrastructure-loki.md - OpenStack: - openstack-overview.md From 5a53b10023d6db7cea5ce32da04fa413d8f69dbd Mon Sep 17 00:00:00 2001 From: Kevin Carter Date: Tue, 17 Dec 2024 10:30:52 -0600 Subject: [PATCH 2/3] fix: removed the grafana readme file This change removes the grafana README file because this content is old, doesn't work, and largely duplicates what we already have in our main docs. Signed-off-by: Kevin Carter --- base-helm-configs/grafana/README.md | 125 ---------------------------- 1 file changed, 125 deletions(-) delete mode 100644 base-helm-configs/grafana/README.md diff --git a/base-helm-configs/grafana/README.md b/base-helm-configs/grafana/README.md deleted file mode 100644 index 6558e361..00000000 --- a/base-helm-configs/grafana/README.md +++ /dev/null @@ -1,125 +0,0 @@ -# Install the grafana helm chart - ---- - -## Recommended to create secret client file instead of putting info in cli - -Example secret file: - -``` -apiVersion: v1 -data: - client_id: BASE64_ENCODED_CLIENT_ID - client_secret: BASE64_ENCODED_CLIENT_SECRET -kind: Secret -metadata: - name: azure-client - namespace: grafana -type: Opaque -``` - ---- - -## Create a datasources yaml file to prepopulate grafana with your datasources. - -example datasources yaml file: - -``` -datasources: - datasources.yaml: - apiVersion: 1 - datasources: - - name: Prometheus - type: prometheus - access: proxy - url: http://kube-prometheus-stack-prometheus.prometheus.svc.cluster.local:9090 - isDefault: true - - name: Loki - type: loki - access: proxy - url: http://loki-gateway.{{ .Release.Namespace }}.svc.cluster.local:80 - editable: false -``` - ---- - -## Create your ssl files - -Your cert file should look something like the following. - -example cert file: - -``` ------BEGIN CERTIFICATE----- -MIID0DCCArigAwIBAgIBATANBgkqhkiG9w0BAQUFADB/MQswCQYDVQQGEwJGUjET -MBEGA1UECAwKU29tZS1TdGF0ZTEOMAwGA1UEBwwFUGFyaXMxDTALBgNVBAoMBERp -bWkxDTALBgNVBAsMBE5TQlUxEDAOBgNVBAMMB0RpbWkgQ0ExGzAZBgkqhkiG9w0B -CQEWDGRpbWlAZGltaS5mcjAeFw0xNDAxMjgyMDM2NTVaFw0yNDAxMjYyMDM2NTVa -MFsxCzAJBgNVBAYTAkZSMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJ -bnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQxFDASBgNVBAMMC3d3dy5kaW1pLmZyMIIB -IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvpnaPKLIKdvx98KW68lz8pGa -RRcYersNGqPjpifMVjjE8LuCoXgPU0HePnNTUjpShBnynKCvrtWhN+haKbSp+QWX -SxiTrW99HBfAl1MDQyWcukoEb9Cw6INctVUN4iRvkn9T8E6q174RbcnwA/7yTc7p -1NCvw+6B/aAN9l1G2pQXgRdYC/+G6o1IZEHtWhqzE97nY5QKNuUVD0V09dc5CDYB -aKjqetwwv6DFk/GRdOSEd/6bW+20z0qSHpa3YNW6qSp+x5pyYmDrzRIR03os6Dau -ZkChSRyc/Whvurx6o85D6qpzywo8xwNaLZHxTQPgcIA5su9ZIytv9LH2E+lSwwID -AQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1PcGVuU1NMIEdlbmVy -YXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQU+tugFtyN+cXe1wxUqeA7X+yS3bgw -HwYDVR0jBBgwFoAUhMwqkbBrGp87HxfvwgPnlGgVR64wDQYJKoZIhvcNAQEFBQAD -ggEBAIEEmqqhEzeXZ4CKhE5UM9vCKzkj5Iv9TFs/a9CcQuepzplt7YVmevBFNOc0 -+1ZyR4tXgi4+5MHGzhYCIVvHo4hKqYm+J+o5mwQInf1qoAHuO7CLD3WNa1sKcVUV -vepIxc/1aHZrG+dPeEHt0MdFfOw13YdUc2FH6AqEdcEL4aV5PXq2eYR8hR4zKbc1 -fBtuqUsvA8NWSIyzQ16fyGve+ANf6vXvUizyvwDrPRv/kfvLNa3ZPnLMMxU98Mvh -PXy3PkB8++6U4Y3vdk2Ni2WYYlIls8yqbM4327IKmkDc2TimS8u60CT47mKU7aDY -cbTV5RDkrlaYwm5yqlTIglvCv7o= ------END CERTIFICATE----- -``` - -example key file: - -``` ------BEGIN RSA PRIVATE KEY----- -MIIEowIBAAKCAQEAvpnaPKLIKdvx98KW68lz8pGaRRcYersNGqPjpifMVjjE8LuC -oXgPU0HePnNTUjpShBnynKCvrtWhN+haKbSp+QWXSxiTrW99HBfAl1MDQyWcukoE -b9Cw6INctVUN4iRvkn9T8E6q174RbcnwA/7yTc7p1NCvw+6B/aAN9l1G2pQXgRdY -C/+G6o1IZEHtWhqzE97nY5QKNuUVD0V09dc5CDYBaKjqetwwv6DFk/GRdOSEd/6b -W+20z0qSHpa3YNW6qSp+x5pyYmDrzRIR03os6DauZkChSRyc/Whvurx6o85D6qpz -ywo8xwNaLZHxTQPgcIA5su9ZIytv9LH2E+lSwwIDAQABAoIBAFml8cD9a5pMqlW3 -f9btTQz1sRL4Fvp7CmHSXhvjsjeHwhHckEe0ObkWTRsgkTsm1XLu5W8IITnhn0+1 -iNr+78eB+rRGngdAXh8diOdkEy+8/Cee8tFI3jyutKdRlxMbwiKsouVviumoq3fx -OGQYwQ0Z2l/PvCwy/Y82ffq3ysC5gAJsbBYsCrg14bQo44ulrELe4SDWs5HCjKYb -EI2b8cOMucqZSOtxg9niLN/je2bo/I2HGSawibgcOdBms8k6TvsSrZMr3kJ5O6J+ -77LGwKH37brVgbVYvbq6nWPL0xLG7dUv+7LWEo5qQaPy6aXb/zbckqLqu6/EjOVe -ydG5JQECgYEA9kKfTZD/WEVAreA0dzfeJRu8vlnwoagL7cJaoDxqXos4mcr5mPDT -kbWgFkLFFH/AyUnPBlK6BcJp1XK67B13ETUa3i9Q5t1WuZEobiKKBLFm9DDQJt43 -uKZWJxBKFGSvFrYPtGZst719mZVcPct2CzPjEgN3Hlpt6fyw3eOrnoECgYEAxiOu -jwXCOmuGaB7+OW2tR0PGEzbvVlEGdkAJ6TC/HoKM1A8r2u4hLTEJJCrLLTfw++4I -ddHE2dLeR4Q7O58SfLphwgPmLDezN7WRLGr7Vyfuv7VmaHjGuC3Gv9agnhWDlA2Q -gBG9/R9oVfL0Dc7CgJgLeUtItCYC31bGT3yhV0MCgYEA4k3DG4L+RN4PXDpHvK9I -pA1jXAJHEifeHnaW1d3vWkbSkvJmgVf+9U5VeV+OwRHN1qzPZV4suRI6M/8lK8rA -Gr4UnM4aqK4K/qkY4G05LKrik9Ev2CgqSLQDRA7CJQ+Jn3Nb50qg6hFnFPafN+J7 -7juWln08wFYV4Atpdd+9XQECgYBxizkZFL+9IqkfOcONvWAzGo+Dq1N0L3J4iTIk -w56CKWXyj88d4qB4eUU3yJ4uB4S9miaW/eLEwKZIbWpUPFAn0db7i6h3ZmP5ZL8Q -qS3nQCb9DULmU2/tU641eRUKAmIoka1g9sndKAZuWo+o6fdkIb1RgObk9XNn8R4r -psv+aQKBgB+CIcExR30vycv5bnZN9EFlIXNKaeMJUrYCXcRQNvrnUIUBvAO8+jAe -CdLygS5RtgOLZib0IVErqWsP3EI1ACGuLts0vQ9GFLQGaN1SaMS40C9kvns1mlDu -LhIhYpJ8UsCVt5snWo2N+M+6ANh5tpWdQnEK6zILh4tRbuzaiHgb ------END RSA PRIVATE KEY----- -``` - ---- - -## Add repo and install - -``` -helm repo add grafana https://grafana.github.io/helm-charts -helm repo update -kubectl create ns grafana -kubectl -n grafana create secret tls grafana-tls-public --cert=YOUR_CERT_FILE --key=YOUR_KEY_FILE - -kubectl -n grafana create secret generic azure-client --type Opaque --from-literal=client_id="YOUR_CLIENT_ID" --from-literal=client_secret="YOUR_CLIENT_SECRET" -OR -kubectl -n grafana apply -f azure-secrets.yaml - -helm upgrade --install grafana grafana/grafana --namespace grafana --values overrides.yaml -f datasources.yaml --set tenant_id=YOUR_TENANT_ID --set custom_host=YOUR_URL_FOR_INGRESS -``` From 31695c6587889ea896db77264e3d1654e47f7880 Mon Sep 17 00:00:00 2001 From: Kevin Carter Date: Tue, 17 Dec 2024 10:42:23 -0600 Subject: [PATCH 3/3] fix: cleanup loki docs so that our information is more consistent Signed-off-by: Kevin Carter --- ...nfrastructure-fluentd.md => infrastructure-fluentbit.md} | 0 docs/infrastructure-loki.md | 6 ++++-- 2 files changed, 4 insertions(+), 2 deletions(-) rename docs/{infrastructure-fluentd.md => infrastructure-fluentbit.md} (100%) diff --git a/docs/infrastructure-fluentd.md b/docs/infrastructure-fluentbit.md similarity index 100% rename from docs/infrastructure-fluentd.md rename to docs/infrastructure-fluentbit.md diff --git a/docs/infrastructure-loki.md b/docs/infrastructure-loki.md index 83b7e8a3..3067c03e 100644 --- a/docs/infrastructure-loki.md +++ b/docs/infrastructure-loki.md @@ -1,5 +1,7 @@ # Setting up Loki +Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost-effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream. + ## Add the grafana helm repo ``` shell @@ -9,11 +11,11 @@ helm repo update ### Install the helm chart -You will need to make changes depending on how you want to configure loki. Example files are included in this directory choose one relevant to your deploy +ou will need to make changes depending on how you want to configure loki. Example files are included in `genetack/base-helm-configs`. Choose one relevant to your deploy, edit for revelant data, and ensure you copy the file to `/etc/genestack/base-helm/loki-helm-overrides.yaml` ``` shell helm upgrade --install \ - --values my-loki-helm-overrides.yaml \ + --values /etc/genestack/helm-configs/loki/loki-helm-overrides.yaml \ loki grafana/loki \ --create-namespace \ --namespace grafana \