Load balancer reconcile stuck waiting for LB to join private network

[lhp-nemlig]

TL;DR

The hetzner load-balancer is currently prone to getting stuck on "wait_until_finished" after calling "attach_to_network".

Expected behavior

The API should return return status = success when the load balancer has joined the network, resulting in the cloud controller manager to begin adding targets to the load-balancer.

Observed behavior

The cloud controller manager waits a long time (more than 5 minutes has been observed) for the load balancer to report that it has finished joining the private network, resulting in other problems, such as helm timeout if helm is waiting for the load balancer to be provisioned.

This can be fixed by repeatedly killing the hcloud-cloud-controller-manager pods, I assume this causes the reconciliation loop to realize that the load balancer is already attached and does not need to be "re-attached", therefore proceeding to target assignment.

Minimal working example

No response

Log output

No response

Additional information

This also happens if using the hcloud python library, It is a general Hetzner API issue, not hcloud-go specific.

This might be more relevant for the Hetzner internal API team, but this was the best place I could think of reporting it. hcloud-cloud-controller-manager might also be able to workaround the problem.