Connection Issues: Thermostats become unavailable randomly after HA OS update

[LXR85]

I have 5 CC-RT-BLE-EQ thermostats, which have worked fine for many months now, thanks to your efforts. After updating to Home Assistant OS 10 two weeks ago, I have a lot of intermittent connection issues. I am using the built-in Bluetooth adapter from the Raspberry Pi 3B. The following error message comes up in the logs:

This error originated from a custom integration.

Logger: custom_components.dbuezas_eq3btsmart.climate
Source: custom_components/dbuezas_eq3btsmart/climate.py:135
Integration: eQ-3 Bluetooth Smart Thermostats (dbuezas) (documentation, issues)
First occurred: 18:33:25 (4 occurrences)
Last logged: 18:41:47

[kitchen] Error updating:

There is a traceback with error messages related to this integration:

Traceback (most recent call last):
File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
await fut
asyncio.exceptions.CancelledError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 456, in wait_for
return fut.result()
asyncio.exceptions.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/config/custom_components/dbuezas_eq3btsmart/sensor.py", line 122, in fetch_serial
await self._thermostat.async_query_id()
File "/config/custom_components/dbuezas_eq3btsmart/python_eq3bt/eq3bt/eq3btsmart.py", line 160, in async_query_id
await self._conn.async_make_request(value)
File "/config/custom_components/dbuezas_eq3btsmart/python_eq3bt/eq3bt/bleakconnection.py", line 164, in async_make_request
await self._async_make_request_try(value, retries)
File "/config/custom_components/dbuezas_eq3btsmart/python_eq3bt/eq3bt/bleakconnection.py", line 203, in _async_make_request_try
raise ex
File "/config/custom_components/dbuezas_eq3btsmart/python_eq3bt/eq3bt/bleakconnection.py", line 182, in _async_make_request_try
await asyncio.wait_for(
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 458, in wait_for
raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError

I have already fully rebooted and power-disconnected the Pi a few times, to no avail. According to the changelog, they updated the BlueZ package in Home Assistant OS 10:

Updated software packages
[...] The latest version of the Bluetooth package BlueZ 5.66 comes with various bug fixes for improved communication with Bluetooth devices. [...]

Here is my System Information report:

## System Information

version | core-2023.5.2
-- | --
installation_type | Home Assistant OS
dev | false
hassio | true
docker | true
user | root
virtualenv | false
python_version | 3.10.11
os_name | Linux
os_version | 6.1.21-v7
arch | armv7l
timezone | Europe/Berlin
config_dir | /config

<details><summary>Home Assistant Community Store</summary>

GitHub API | ok
-- | --
GitHub Content | ok
GitHub Web | ok
GitHub API Calls Remaining | 4929
Installed Version | 1.32.1
Stage | running
Available Repositories | 1285
Downloaded Repositories | 8

</details>

<details><summary>Home Assistant Cloud</summary>

logged_in | false
-- | --
can_reach_cert_server | ok
can_reach_cloud_auth | ok
can_reach_cloud | ok

</details>

<details><summary>Home Assistant Supervisor</summary>

host_os | Home Assistant OS 10.1
-- | --
update_channel | stable
supervisor_version | supervisor-2023.04.1
agent_version | 1.5.1
docker_version | 23.0.3
disk_total | 28.6 GB
disk_used | 10.4 GB
healthy | true
supported | true
board | rpi3
supervisor_api | ok
version_api | ok
installed_addons | File editor (5.6.0), Let's Encrypt (4.12.8), Terminal & SSH (9.7.0), ADB - Android Debug Bridge (0.7.0), Duck DNS (1.15.0), Spotify Connect (0.12.3), MPD (1.7.2), Mosquitto broker (6.2.1), Zigbee2MQTT (1.30.4-1), Samba share (10.0.1)

</details>

<details><summary>Dashboards</summary>

dashboards | 1
-- | --
resources | 4
views | 1
mode | storage

</details>

<details><summary>Recorder</summary>

oldest_recorder_run | May 4, 2023 at 00:18
-- | --
current_recorder_run | May 8, 2023 at 18:27
estimated_db_size | 49.25 MiB
database_engine | sqlite
database_version | 3.40.1

</details>

[dbuezas]

There seem to be some issues with Bluetooth in home assistant 10. I thought it was because the drivers for my Broadcom dongle are missing. Which Bluetooth dongle do you have?

[LXR85]

I am using the built-in Bluetooth adapter from the Raspberry Pi 3B.

According to the Raspberry Pi 3B specs page its this one:

BCM43438 wireless LAN and Bluetooth Low Energy (BLE) on board

[dbuezas]

Broadcom. May be also affected by this.
If you ssh into the server and run this:

dmesg | grep -i bluetooth

You can check.

If you see a line like [ 355.837213] Bluetooth: hci1: BCM: firmware Patch file not found, tried:, it means the dongle is missing drivers

[LXR85]

# dmesg | grep -i bluetooth
[   16.824735] Bluetooth: Core ver 2.22
[   16.824883] NET: Registered PF_BLUETOOTH protocol family
[   16.824890] Bluetooth: HCI device and connection manager initialized
[   16.824909] Bluetooth: HCI socket layer initialized
[   16.824920] Bluetooth: L2CAP socket layer initialized
[   16.824938] Bluetooth: SCO socket layer initialized
[   16.841991] Bluetooth: HCI UART driver ver 2.3
[   16.842015] Bluetooth: HCI UART protocol H4 registered
[   16.842094] Bluetooth: HCI UART protocol Three-wire (H5) registered
[   16.842227] Bluetooth: HCI UART protocol Broadcom registered
[   17.478964] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   17.478986] Bluetooth: BNEP filters: protocol multicast
[   17.479006] Bluetooth: BNEP socket layer initialized
[   17.482511] Bluetooth: MGMT ver 1.22
[   81.457068] Bluetooth: RFCOMM TTY layer initialized
[   81.457110] Bluetooth: RFCOMM socket layer initialized
[   81.457139] Bluetooth: RFCOMM ver 1.11
[  323.356961] Bluetooth: hci0: Opcode 0x200d failed: -110
[  323.357137] Bluetooth: hci0: request failed to create LE connection: err -110
[ 7423.546014] Bluetooth: hci0: Frame reassembly failed (-84)
[ 7423.546216] Bluetooth: hci0: Frame reassembly failed (-84)
[ 7423.548329] Bluetooth: hci0: Frame reassembly failed (-84)
[ 9299.509635] Bluetooth: hci0: Frame reassembly failed (-84)
[ 9299.511511] Bluetooth: hci0: Frame reassembly failed (-84)
[ 9299.567820] Bluetooth: hci0: Frame reassembly failed (-84)
[ 9299.567845] Bluetooth: hci0: Received unexpected HCI Event 0x00
[ 9299.569437] Bluetooth: hci0: Frame reassembly failed (-84)
[ 9299.614798] Bluetooth: hci0: Received unexpected HCI Event 0x00
[ 9299.614934] Bluetooth: hci0: Frame reassembly failed (-84)
[ 9299.616605] Bluetooth: hci0: Frame reassembly failed (-84)
[ 9363.452002] Bluetooth: hci0: Frame reassembly failed (-84)
[ 9363.453522] Bluetooth: hci0: Frame reassembly failed (-84)
[10965.038182] Bluetooth: hci0: Frame reassembly failed (-84)
[10965.039638] Bluetooth: hci0: Frame reassembly failed (-84)
[11335.814259] Bluetooth: hci0: Frame reassembly failed (-84)
[11335.815428] Bluetooth: hci0: Frame reassembly failed (-84)

[LXR85]

This issue seems to be related:

home-assistant/operating-system#2485

[zigt1301]

I'm having the same issue right now with my Raspberry Pi 3b+. Two EQ3BT thermostats were running fine. But then I updated HAOS to 11.2 (I don't know which version I was running before that) and for some reason HAOS would not boot. So I did a rollback on HAOS to 11.1. It would then boot, which was a relief.
But now my thermostats are unavailable, even though my BT device_trackers work fine. Rolling back HAOS to 11.0 did not help. Rolling back to HAOS 10.5 did not work. I also tried to remove the BT adapter integration and the EQ3BT integrations, no success. Reinstalling EQ3BT in HACS did not solve it.

Anybody know how to deal with this situation?

[dbuezas]

Try a hard reboot (power cycle the machine) or run this script https://gist.github.com/dbuezas/74c18293c22e6e31bfeb332fa826527f

I put this in an automation via https://www.home-assistant.io/integrations/shell_command/
It runs if a thermostat is unavailable for 10 minutes