Allow certain .kubernetes.io annotations to be managed

[joey-squid]

Description

Feature #746 added an ignore_annotations configuration. Unfortunately, the annotations in this list are added to the immutable set of annotations in isInternalKey. This means that there is no way to tell the provider that e.g. the alb.ingress.kubernetes.io/* annotations should be managed. (There's an ALB-specific exception in isInternalKey but it seems to be outdated.)

Potential Terraform Configuration

I use CDKTF so don't have tested HCL handy, but here's some JSON: (I deleted the very long ingress spec)

"kubernetes_ingress_v1": {
  "ingress": {
    "metadata": {
      "annotations": {
        "alb.ingress.kubernetes.io/certificate-arn": "<redacted>",
        "alb.ingress.kubernetes.io/healthcheck-interval-seconds": "15",
        "alb.ingress.kubernetes.io/healthcheck-path": "/heartbeat",
        "alb.ingress.kubernetes.io/healthcheck-timeout-seconds": "10",
        "alb.ingress.kubernetes.io/load-balancer-attributes": "idle_timeout.timeout_seconds=300",
        "alb.ingress.kubernetes.io/load-balancer-name": "core-wildcard-alb",
        "alb.ingress.kubernetes.io/scheme": "internet-facing",
        "alb.ingress.kubernetes.io/target-type": "ip",
        "alb.ingress.kubernetes.io/wafv2-acl-arn": "<redacted>"
      },
      "name": "core-wildcard-alb",
      "namespace": "default"
    }
  }
}

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[joey-squid]

Implementation thoughts

Given #746 (and PR #1687), it should be possible to remove the removeInternalKeys function altogether, and simply have a default value for ignore_annotations and ignore_labels that reflects the rules in that function. The downside is that anyone using ignore_annotations or ignore_labels would have to update their provider configs to add the default value back in, otherwise the user-specified value overrides the default value instead of being in addition to it (which is exactly the feature I'm after).

I'm not sure if it is a good idea to do it that way as that is a breaking change for anyone who uses that feature already; the alternative would be to add additional boolean fields (ignore_internal_annotations resp. _labels), which doesn't have that problem since it's a new field entirely.

With guidance, I'd be happy to work on either of these solutions; they seem fairly straightforward to implement and would solve a huge headache for me.

[joey-squid]

I've been playing with this a bit and it doesn't seem like a default ignore_annotations is feasible with Go's regex library (it doesn't support negative lookaheads, without which ".kubernetes.io except service.beta.kubernetes.io" is technically possible but unwieldy). Would an ignore_internal_annotations boolean make sense, then?