CCM could receive invalid feature gate option

[Kostov6]

How to categorize this issue?

/area quality
/kind bug
/platform aws

What happened:
On a cluster I observed the following:

$ k logs cloud-controller-manager-... | head -n 1
Error: invalid argument "RotateKubeletServerCertificate=true" for "--feature-gates" flag: unrecognized feature gate: RotateKubeletServerCertificate

$ k logs cloud-controller-manager-... | grep -A 26 'feature-gates '
    --feature-gates mapStringBool              A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
                                               APIResponseCompression=true|false (BETA - default=true)
                                               APIServerIdentity=true|false (BETA - default=true)
                                               APIServerTracing=true|false (BETA - default=true)
                                               AdmissionWebhookMatchConditions=true|false (BETA - default=true)
                                               AggregatedDiscoveryEndpoint=true|false (BETA - default=true)
                                               AllAlpha=true|false (ALPHA - default=false)
                                               AllBeta=true|false (BETA - default=false)
                                               CloudControllerManagerWebhook=true|false (ALPHA - default=false)
                                               CloudDualStackNodeIPs=true|false (BETA - default=true)
                                               ComponentSLIs=true|false (BETA - default=true)
                                               ConsistentListFromCache=true|false (ALPHA - default=false)
                                               ContextualLogging=true|false (ALPHA - default=false)
                                               InPlacePodVerticalScaling=true|false (ALPHA - default=false)
                                               LoggingAlphaOptions=true|false (ALPHA - default=false)
                                               LoggingBetaOptions=true|false (BETA - default=true)
                                               OpenAPIEnums=true|false (BETA - default=true)
                                               StableLoadBalancerNodeSet=true|false (BETA - default=true)
                                               StorageVersionAPI=true|false (ALPHA - default=false)
                                               StorageVersionHash=true|false (BETA - default=true)
                                               StructuredAuthenticationConfiguration=true|false (ALPHA - default=false)
                                               StructuredAuthorizationConfiguration=true|false (ALPHA - default=false)
                                               UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
                                               ValidatingAdmissionPolicy=true|false (BETA - default=false)
                                               WatchList=true|false (ALPHA - default=false)
                                               ZeroLimitedNominalConcurrencyShares=true|false (BETA - default=false)
    --kube-api-burst int32                     Burst to use while talking with kubernetes apiserver. (default 30)

What you expected to happen:
The feature gate validation to have rejected this cluster

How to reproduce it (as minimally and precisely as possible):
The following test is wrong because this is not a CCM feature gate:

gardener-extension-provider-aws/pkg/apis/aws/validation/controlplane_test.go

Line 35 in aaf1ad5

"AnyVolumeDataSource": true,

Also another feature gate that should fail but passes

Describe("#ValidateControlPlaneConfig", func() {
		It("should fail with invalid RotateKubeletServerCertificate CCM feature gates but passes", func() {
			controlPlane.CloudControllerManager = &apisaws.CloudControllerManagerConfig{
				FeatureGates: map[string]bool{
					"RotateKubeletServerCertificate": true,
				},
			}

			errorList := ValidateControlPlaneConfig(controlPlane, "1.28.3", fldPath)

			Expect(errorList).To(BeEmpty())
		})

Anything else we need to know?:

Environment:

• Gardener version (if relevant):
• Extension version:
• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• Others: