If heath check starts failing it is not propagated back to DNSRecord/DNSPolicy

[trepel]

Let's have DNSPolicy with health check configured being successfully enforced, DNSRecord saying "All healthchecks succeeded" and DNSHealthCheckProbe's .status.healthy saying "true". If this is the case and health check starts failing (for whatever reason afaict) then the DNSHealthCheckProbe's .status.healthy starts saying "false" (expected) but neither DNSPolicy nor DNSRecord are updated accordingly. Their statuses remain unchanged. DNS Operator needs to be restarted for these two to get updated.

I tried to make health check fail by

• passing non-existent secret name to additionalHeadersRef
• removing additionalHeadersRef
• scaling down the backend

It made no difference.

Reverse scenario - making the health check pass when all three DNS* CRs are not happy - works, no need to restart the DNS Operator, all three CRs get updated as expected.

Tested with v0.10.0 as part of Kuadrant v1.0.0-rc8

[maksymvavilov]

The premature reconciliation on the probe watch does not trigger if changing healthy -> unhealthy. Prob has something to do with default values in the watch