Help needed with updating reposerver and making kbld work

[AvinashRamireddy31]

I followed this tutorial
I am trying to use kbld for argocd application. But I am getting error.

Ran below steps on minikube on Mac m1 air

minikube version
minikube version: v1.31.1
commit: fd3f3801765d093a485d255043149f92ec0a695f

helm repo add argo https://argoproj.github.io/argo-helm
helm upgrade -i  my-argo-cd argo/argo-cd --version 4.5.4 -f argocd-values.yaml

Values file(argocd-values.yaml) contain following

dex:
  enabled: false
notifications:
  enabled: false
applicationSet:
  enabled: false
server:
  extraArgs:
    - --insecure
  config:
    configManagementPlugins: |
      - name: kbld
        generate:
          command: ["bash", "-c"]
          args: ['helm template --release-name "$ARGOCD_APP_NAME" -f <(echo "$HELM_VALUES") . > kbld.yaml && kbld -f kbld.yaml >> final.yaml && cat final.yaml']
repoServer:
  initContainers:
    - name: download-tools
      image: busybox:1.35.0
      command: [ sh, -c ]
      env:
        - name: KBLD_VERSION
          value: "0.33.0"
      args:
        - wget -q https://github.com/vmware-tanzu/carvel-kbld/releases/download/v${KBLD_VERSION}/kbld-linux-amd64 &&
          mv kbld-linux-amd64 /custom-tools/kbld &&
          chmod +x /custom-tools/kbld
      volumeMounts:
        - mountPath: /custom-tools
          name: custom-tools
  volumes:
    - name: custom-tools
      emptyDir: { }
  volumeMounts:
    - mountPath: /usr/local/bin/kbld
      name: custom-tools
      subPath: kbld

Following is the pod log

k describe pod my-argo-cd-argocd-repo-server-6676b45448-l2lvg
Name:             my-argo-cd-argocd-repo-server-6676b45448-l2lvg
Namespace:        default
Priority:         0
Service Account:  my-argo-cd-argocd-repo-server
Node:             minikube/192.168.105.4
Start Time:       Mon, 24 Jun 2024 00:06:45 +0530
Labels:           app.kubernetes.io/component=repo-server
                  app.kubernetes.io/instance=my-argo-cd
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=argocd-repo-server
                  app.kubernetes.io/part-of=argocd
                  app.kubernetes.io/version=v2.3.3
                  helm.sh/chart=argo-cd-4.5.4
                  pod-template-hash=6676b45448
Annotations:      <none>
Status:           Pending
IP:               10.244.0.20
IPs:
  IP:           10.244.0.20
Controlled By:  ReplicaSet/my-argo-cd-argocd-repo-server-6676b45448
Init Containers:
  copyutil:
    Container ID:  docker://0e6ac7e96fa727354e2d5243fd9b5c4150b4db31bd16971d8f75630230bfb086
    Image:         quay.io/argoproj/argocd:v2.3.3
    Image ID:      docker-pullable://quay.io/argoproj/argocd@sha256:dd738f234fcdb0aac8631a0fd1aafbbcd86f936480b06e8377b033ef7a764f71
    Port:          <none>
    Host Port:     <none>
    Command:
      cp
      -n
      /usr/local/bin/argocd
      /var/run/argocd/argocd-cmp-server
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Mon, 24 Jun 2024 00:06:46 +0530
      Finished:     Mon, 24 Jun 2024 00:06:46 +0530
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/argocd from var-files (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-whmmz (ro)
  download-tools:
    Container ID:  docker://6ebd3624491c23302f2fa7ecb4f8b307e88cf96a8ee682eae5bda2a1a9cd7585
    Image:         busybox:1.35.0
    Image ID:      docker-pullable://busybox@sha256:5be7104a4306abe768359a5379e6050ef69a29e9a5f99fcf7f46d5f7e9ba29a2
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
    Args:
      wget -q https://github.com/vmware-tanzu/carvel-kbld/releases/download/v${KBLD_VERSION}/kbld-linux-amd64 && mv kbld-linux-amd64 /custom-tools/kbld && chmod +x /custom-tools/kbld
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Mon, 24 Jun 2024 00:12:41 +0530
      Finished:     Mon, 24 Jun 2024 00:12:42 +0530
    Ready:          False
    Restart Count:  6
    Environment:
      KBLD_VERSION:  0.33.0
    Mounts:
      /custom-tools from custom-tools (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-whmmz (ro)
Containers:
  repo-server:
    Container ID:  
    Image:         quay.io/argoproj/argocd:v2.3.3
    Image ID:      
    Port:          8081/TCP
    Host Port:     0/TCP
    Command:
      entrypoint.sh
    Args:
      argocd-repo-server
      --redis
      my-argo-cd-argocd-redis:6379
      --logformat
      text
      --loglevel
      info
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Liveness:       tcp-socket :8081 delay=10s timeout=1s period=10s #success=1 #failure=3
    Readiness:      tcp-socket :8081 delay=10s timeout=1s period=10s #success=1 #failure=3
    Environment:
      HELM_CACHE_HOME:   /helm-working-dir
      HELM_CONFIG_HOME:  /helm-working-dir
      HELM_DATA_HOME:    /helm-working-dir
    Mounts:
      /app/config/gpg/keys from gpg-keyring (rw)
      /app/config/gpg/source from gpg-keys (rw)
      /app/config/reposerver/tls from argocd-repo-server-tls (rw)
      /app/config/ssh from ssh-known-hosts (rw)
      /app/config/tls from tls-certs (rw)
      /helm-working-dir from helm-working-dir (rw)
      /home/argocd/cmp-server/plugins from plugins (rw)
      /tmp from tmp-dir (rw)
      /usr/local/bin/kbld from custom-tools (rw,path="kbld")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-whmmz (ro)
Conditions:
  Type              Status
  Initialized       False 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  custom-tools:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  gpg-keys:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      argocd-gpg-keys-cm
    Optional:  false
  gpg-keyring:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  ssh-known-hosts:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      argocd-ssh-known-hosts-cm
    Optional:  false
  tls-certs:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      argocd-tls-certs-cm
    Optional:  false
  helm-working-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  argocd-repo-server-tls:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  argocd-repo-server-tls
    Optional:    true
  tmp-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  var-files:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  plugins:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  kube-api-access-whmmz:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                     From               Message
  ----     ------     ----                    ----               -------
  Normal   Scheduled  7m49s                   default-scheduler  Successfully assigned default/my-argo-cd-argocd-repo-server-6676b45448-l2lvg to minikube
  Normal   Pulled     7m48s                   kubelet            Container image "quay.io/argoproj/argocd:v2.3.3" already present on machine
  Normal   Created    7m48s                   kubelet            Created container copyutil
  Normal   Started    7m48s                   kubelet            Started container copyutil
  Normal   Started    7m3s (x4 over 7m47s)    kubelet            Started container download-tools
  Normal   Pulled     6m11s (x5 over 7m47s)   kubelet            Container image "busybox:1.35.0" already present on machine
  Normal   Created    6m11s (x5 over 7m47s)   kubelet            Created container download-tools
  Warning  BackOff    2m46s (x25 over 7m45s)  kubelet            Back-off restarting failed container download-tools in pod my-argo-cd-argocd-repo-server-6676b45448-l2lvg_default(5cbd789a-a1a2-4cc0-a082-26a6ab66127d)

[joaopapereira]

From a cursory look, I could not see any error in your output. Can you check what error you are getting?
Also, the URL was changed to https://github.com/carvel-dev/kbld/ instead of tanzu-vmware. Not sure if that affects something because there will be a redirect.

[AvinashRamireddy31]

In the last line it says Back-off restarting failed container download-tools

[joaopapereira]

Hmm, I need more information. What is the error that you get inside the container? Something is happening inside the container that is causing this container to fail, and we need first to understand that before we can better diagnose the problem