Error: failed to install CRD crds/khcheck.yaml: resource mapping not found for name: "khchecks.comcast.github.io" namespace: "" from "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1" ensure CRDs are installed first

[nisargtk]

Getting below error while installing it on jenkins with secrets manager. Though to skip kuberhealthy but it gives other error from kuberhealthy

Initial error

Error: failed to install CRD crds/khcheck.yaml: resource mapping not found for name: "khchecks.comcast.github.io" namespace: "" from "": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
│ ensure CRDs are installed first

After disabling kuberhealthy with install_kuberhealthy = false getting below error.

error validating "config-root/namespaces/jx/jx-kh-check-health-checks-jx/jx-bot-token-kuberhealthycheck.yaml": error validating data: ValidationError(KuberhealthyCheck.spec.podSpec.containers[0]): unknown field "restartPolicy" in io.github.comcast.v1.KuberhealthyCheck.spec.podSpec.containers; if you choose to ignore these errors, turn validation off with --validate=false
error validating "config-root/namespaces/jx/jx-kh-check-health-checks-jx/jx-webhook-events-kuberhealthycheck.yaml": error validating data: ValidationError(KuberhealthyCheck.spec.podSpec.containers[0]): unknown field "restartPolicy" in io.github.comcast.v1.KuberhealthyCheck.spec.podSpec.containers; if you choose to ignore these errors, turn validation off with --validate=false
error validating "config-root/namespaces/jx/jx-kh-check-health-checks-jx/jx-webhook-kuberhealthycheck.yaml": error validating data: ValidationError(KuberhealthyCheck.spec.podSpec.containers[0]): unknown field "restartPolicy" in io.github.comcast.v1.KuberhealthyCheck.spec.podSpec.containers; if you choose to ignore these errors, turn validation off with --validate=false
make[1]: *** [versionStream/src/Makefile.mk:322: kubectl-apply] Error 1
make[1]: Leaving directory '/workspace/source'
error: failed to regenerate: failed to regenerate phase 1: failed to run 'make regen-phase-1 NEW_CLUSTER=false' command in directory '.', output: ''
make: *** [versionStream/src/Makefile.mk:269: regen-check] Error 1

main.tf
module "eks-jx" {
  source               = "github.com/jenkins-x/terraform-aws-eks-jx?ref=v1.21.0"
  cluster_name         = data.aws_eks_cluster.cluster.id
  region               = var.region
  is_jx2               = false
  jx_git_url           = var.jx_git_url
  jx_bot_username      = var.jx_bot_username
  jx_bot_token         = var.jx_bot_token
  force_destroy        = var.force_destroy
  nginx_chart_version  = var.nginx_chart_version
  install_kuberhealthy = false
  create_autoscaler_role = false
  use_vault = false
  use_asm  = true
  create_asm_role = true
  create_eks = false
  create_vpc  = false
  cluster_version = "1.23"
  profile = "eks"
}

[dbason]

This is because apiextensions.k8s.io/v1beta1 has been removed as of kubernetes 1.22. As a work around you could use k8s v1.21, but I'm currently reviewing the terraform to see what could be done. Kuberhealthy v2.7.0 has updated the CRD definitions

[cnukwas]

@nisargtk how did you resolve this as we're running into exact issue?
@dbason we're running into similar issues with EKS and k8s version 1.24. didn't help even after updating kuberhealthy chart version from 0.64 to 0.90. This is causing issues with JX-3 set up. Are there any workarounds other than using old Kubernetes versions?

Thanks

[cnukwas]

I was able to resolve this error by removing jxgh/jx-kh-check chart entry in the cluster repo, in our case it's jx3-eks-asm/src/main/helmfiles/jx/helmfile.yaml file.