[Bug]: MonitorDiagnosticSetting fields continually reconcile #2

[stevendborrelli]

Is there an existing issue for this?

• I have searched the existing issues

Affected Resource(s)

• insights.azure.upbound.io/v1beta1 - MonitorDiagnosticSetting

Resource MRs required to reproduce the bug

# SPDX-FileCopyrightText: 2024 The Crossplane Authors <https://crossplane.io>
#
# SPDX-License-Identifier: Apache-2.0

apiVersion: insights.azure.upbound.io/v1beta1
kind: MonitorDiagnosticSetting
metadata:
  annotations:
    meta.upbound.io/example-id: insights/v1beta2/monitordiagnosticsetting
  labels:
    testing.upbound.io/example-name: insightsmds
  name: insightsmds
spec:
  forProvider:
    enabledLog:
      - categoryGroup: audit
    #logAnalyticsDestinationType: Dedicated
    metric:
      - category: Transaction
        enabled: false
    name: insightsmds
    storageAccountIdSelector:
      matchLabels:
        testing.upbound.io/example-name: insightsmds   
    targetResourceId: /subscriptions/${data.azurerm_client_config.current.subscription_resource_id}/resourceGroups/insightsmds/providers/Microsoft.Storage/storageAccounts/insightsmds/blobServices/default
---
apiVersion: keyvault.azure.upbound.io/v1beta2
kind: Vault
metadata:
  annotations:
    meta.upbound.io/example-id: insights/v1beta2/monitordiagnosticsetting
  labels:
    testing.upbound.io/example-name: insightsmds
  name: insightsmds
spec:
  forProvider:
    location: West Europe
    purgeProtectionEnabled: false
    resourceGroupNameSelector:
      matchLabels:
        testing.upbound.io/example-name: insightsmds
    skuName: standard
    softDeleteRetentionDays: 7
    tenantId:  ${data.azurerm_client_config.current.tenant_id}

---

apiVersion: azure.upbound.io/v1beta1
kind: ResourceGroup
metadata:
  annotations:
    meta.upbound.io/example-id: insights/v1beta1/monitordiagnosticsetting
  labels:
    testing.upbound.io/example-name: insightsmds
  name: insightsmds
spec:
  forProvider:
    location: West Europe

---

apiVersion: storage.azure.upbound.io/v1beta2
kind: Account
metadata:
  annotations:
    meta.upbound.io/example-id: insights/v1beta2/monitordiagnosticsetting
  labels:
    testing.upbound.io/example-name: insightsmds
  name: insightsmds
spec:
  forProvider:
    accountReplicationType: LRS
    accountTier: Standard
    location: West Europe
    resourceGroupNameSelector:
      matchLabels:
        testing.upbound.io/example-name: insightsmds

Steps to Reproduce

provider-azure-containerservice          True        True      xpkg.upbound.io/upbound/provider-azure-containerservice:v1.4.0      11d
provider-azure-insights                  True        True      xpkg.upbound.io/upbound/provider-azure-insights:v1.4.0              7d15h
provider-azure-keyvault                  True        True      xpkg.upbound.io/upbound/provider-azure-keyvault:v1.4.0              7d15h
provider-azure-network                   True        True      xpkg.upbound.io/upbound/provider-azure-network:v1.4.0               11d
provider-azure-operationalinsights       True        True      xpkg.upbound.io/upbound/provider-azure-operationalinsights:v1.4.0   11h
provider-azure-storage                   True        True      xpkg.upbound.io/upbound/provider-azure-storage:v1.4.0               7d15h
upbound-provider-family-azure            True        True      xpkg.upbound.io/upbound/provider-family-azure:v1.4.0                11d

• Enable debugging on provider-azure-insights
• Apply the manifest
• Review debug logs

What happened?

Constant diff detected across the various log, metric and enabledLog fields. By populating these fields from the atProvider we can suppress the diff calculation

Fields below from the resource:

spec:
  deletionPolicy: Delete
  forProvider:
    enabledLog:
    - categoryGroup: audit
    metric:
    - category: Capacity
      retentionPolicy:
        enabled: false
    - category: Transaction
      retentionPolicy:
        enabled: false
    name: insightsmds
    storageAccountId: /subscriptions/038f2b7c-3265-43b8-8624-c9ad5da610a8/resourceGroups/insightsmds/providers/Microsoft.Storage/storageAccounts/insightsmds
    storageAccountIdRef:
      name: insightsmds
    storageAccountIdSelector:
      matchLabels:
        testing.upbound.io/example-name: insightsmds
    targetResourceId: /subscriptions/038f2b7c-3265-43b8-8624-c9ad5da610a8/resourceGroups/insightsmds/providers/Microsoft.Storage/storageAccounts/insightsmds/blobServices/default
  initProvider: {}
  managementPolicies:
  - '*'
  providerConfigRef:
    name: default
status:
  atProvider:
    enabledLog:
    - category: ""
      categoryGroup: audit
      retentionPolicy:
        days: 0
        enabled: false
    eventhubAuthorizationRuleId: ""
    eventhubName: ""
    id: /subscriptions/<redacted>resourceGroups/insightsmds/providers/Microsoft.Storage/storageAccounts/insightsmds/blobServices/default|insightsmds
    log:
    - category: ""
      categoryGroup: allLogs
      enabled: false
      retentionPolicy:
        days: 0
        enabled: false
    - category: ""
      categoryGroup: audit
      enabled: true
      retentionPolicy:
        days: 0
        enabled: false
    logAnalyticsDestinationType: ""
    logAnalyticsWorkspaceId: ""
    metric:
    - category: Capacity
      enabled: true
      retentionPolicy:
        days: 0
        enabled: false
    - category: Transaction
      enabled: true
      retentionPolicy:
        days: 0
        enabled: false
    name: insightsmds
    storageAccountId: /subscriptions/<redacted>/resourceGroups/insightsmds/providers/Microsoft.Storage/storageAccounts/insightsmds
    targetResourceId: /subscriptions/<redacted>/resourceGroups/insightsmds/providers/Microsoft.Storage/storageAccounts/insightsmds/blobServices/default

Relevant Error Output Snippet

2024-08-21T05:47:59Z	DEBUG	provider-azure	Diff detected	{"uid": "b9a19dca-eee7-4588-a30a-0251d2c226b4", "name": "insightsmds", "gvk": "insights.azure.upbound.io/v1beta1, Kind=MonitorDiagnosticSetting", "instanceDiff": "*terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{\"metric.1083658365.category\":*terraform.ResourceAttrDiff{Old:\"Capacity\", New:\"\", NewComputed:false, NewRemoved:true, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1083658365.enabled\":*terraform.ResourceAttrDiff{Old:\"false\", New:\"false\", NewComputed:false, NewRemoved:true, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1083658365.retention_policy.#\":*terraform.ResourceAttrDiff{Old:\"1\", New:\"0\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1083658365.retention_policy.0.days\":*terraform.ResourceAttrDiff{Old:\"0\", New:\"0\", NewComputed:false, NewRemoved:true, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1083658365.retention_policy.0.enabled\":*terraform.ResourceAttrDiff{Old:\"false\", New:\"false\", NewComputed:false, NewRemoved:true, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1758835704.category\":*terraform.ResourceAttrDiff{Old:\"\", New:\"Capacity\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1758835704.enabled\":*terraform.ResourceAttrDiff{Old:\"\", New:\"true\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1758835704.retention_policy.#\":*terraform.ResourceAttrDiff{Old:\"0\", New:\"1\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1758835704.retention_policy.0.days\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.1758835704.retention_policy.0.enabled\":*terraform.ResourceAttrDiff{Old:\"\", New:\"false\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3444833881.category\":*terraform.ResourceAttrDiff{Old:\"\", New:\"Transaction\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3444833881.enabled\":*terraform.ResourceAttrDiff{Old:\"\", New:\"true\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3444833881.retention_policy.#\":*terraform.ResourceAttrDiff{Old:\"0\", New:\"1\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3444833881.retention_policy.0.days\":*terraform.ResourceAttrDiff{Old:\"\", New:\"\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3444833881.retention_policy.0.enabled\":*terraform.ResourceAttrDiff{Old:\"\", New:\"false\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3789768014.category\":*terraform.ResourceAttrDiff{Old:\"Transaction\", New:\"\", NewComputed:false, NewRemoved:true, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3789768014.enabled\":*terraform.ResourceAttrDiff{Old:\"false\", New:\"false\", NewComputed:false, NewRemoved:true, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3789768014.retention_policy.#\":*terraform.ResourceAttrDiff{Old:\"1\", New:\"0\", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3789768014.retention_policy.0.days\":*terraform.ResourceAttrDiff{Old:\"0\", New:\"0\", NewComputed:false, NewRemoved:true, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, \"metric.3789768014.retention_policy.0.enabled\":*terraform.ResourceAttrDiff{Old:\"false\", New:\"false\", NewComputed:false, NewRemoved:true, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, RawConfig:cty.NilVal, RawState:cty.NilVal, RawPlan:cty.NilVal, Meta:map[string]interface {}(nil)}"}

Crossplane Version

1.16.0

Provider Version

1.4.0

Kubernetes Version

No response

Kubernetes Distribution

No response

Additional Info

This is similar to issue #797, but it affects fields that changed in the v1beta1->v1beta2 migration.

[turkenf]

Hi @stevendborrelli, thank you for the issue report.

I could not reproduce the issue. When I apply the manifests above, I get the following error:

    message: 'create failed: async create failed: failed to create the resource: [{0
      creating Monitor Diagnostics Setting "insightsmds" for Resource "/subscriptions/038f2b7c-3265-43b8-8624-c9ad5da610a8/resourceGroups/insightsmds/providers/Microsoft.KeyVault/vaults/insightsmds":
      unexpected status 400 with response: {"code":"BadRequest","message":"Metric
      category ''Transaction'' is not supported, supported categories are: ''AllMetrics''."}  []}]'

After setting the category: AllMetrics I could not face any issues or diff.

[github-actions]

This provider repo does not have enough maintainers to address every issue. Since there has been no activity in the last 90 days it is now marked as stale. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[github-actions]

This issue is being closed since there has been no activity for 14 days since marking it as stale. If you still need help, feel free to comment or reopen the issue!