Allow to define resources for miscellaneous addons

[elliotdobson]

/kind feature

1. Describe IN DETAIL the feature/behavior/change you would like to see.

You are able to specify custom resources on most Kubernetes controllers now in kOps (eg. kubeAPIServer, kubeControllerManager, kubeDNS, kubeProxy, kubeScheduler & etcdClusters).

I'd also like to be able to specify custom resources for any addons that are enabled. Some do already support it like cluster-autoscaler however it is not consistent across them.

Specifically I'd like to see support added for:

• aws-cloud-controller-manager
• calico-kube-controllers & calico-node
• cert-manager, cert-manager-cainjector & cert-manager-webhook
• ebs-csi-controller & ebs-csi-node
• pod-identity-webhook

I'm happy to implement if maintainers agree with adding this feature.

2. Feel free to provide a design supporting your feature request.

There seems to be an existing pattern for including custom resources in the kOps API:

	// CPURequest, cpu request compute resource for kube-controler-manager. Defaults to "100m"
	CPURequest *resource.Quantity `json:"cpuRequest,omitempty"`
	// CPULimit, cpu limit compute resource for kube-controler-manager e.g. "500m"
	CPULimit *resource.Quantity `json:"cpuLimit,omitempty"`
	// MemoryRequest, memory request compute resource for kube-controler-manager e.g. "30Mi"
	MemoryRequest *resource.Quantity `json:"memoryRequest,omitempty"`
	// MemoryLimit, memory limit compute resource for kube-controler-manager e.g. "30Mi"
	MemoryLimit *resource.Quantity `json:"memoryLimit,omitempty"`

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale