No Response

No Response in HomeKit

HomeKit sets the No Response state for an accessory, when a request to that accessory times out, or results in an error response. It clears the No Response state, when a request to the accessory results in a valid response. No Response doesn't mean that the accessory isn't currently responding, only that the last request to the accessory didn't result in a valid response.

HomeKit Accessory Protocol (HAP)

A HomeKit accessory provides a web server to HomeKit, for HomeKit to interact with the accessory. They communicate using the HomeKit Accessory Protocol (HAP), which is based on HTTP. Bluntly speaking, HAP provides three operations on characteristics:

• read: HomeKit issues a request to get the current value, the accessory responds with the value or with an error;
• write: HomeKit issues a request to set a new value; the accessory response with the status (success or error) and, optionally, with the new value;
• event: The accessory pushes a new value to HomeKit, or rather, to HomeKit runtime instances that subscribed to notifications for this characteristic.

When a read or write results in an error or in no response at all, HomeKit sets the No Response state for the accessory, and Home shows No Response in the tile(s). Other HomeKit apps might show this state in a different way, e.g. Eve shows a little red triangle.
When a subsequent read or write succeeds, the No Response state is cleared, and Home no longer shows No Response.
The No Response state isn't updated on an event (I think this is because HomeKit cancels the subscriptions when it sets No Response). So an accessory cannot set not clear No Response, the initiative for this needs to come from HomeKit.

Bridged Accessories

A HomeKit bridge (like Homebridge) provides access to multiple accessories from a single HAP server. When the bridge doesn't respond, HomeKit marks all accessories exposed by that bridge as No Response. By returning an error a bridge can cause No Response to be set for a single bridged accessory, e.g. in case the corresponding device is not reachable. By returning a valid response for the next request, a bridge can cause the No Response state to be cleared. As with non-bridged accessories, there is no way for a bridge to push the No Response state to HomeKit; the initiative needs to come from HomeKit.

Tip: Run Homebridge with DEBUG=* set in the environment, to have HAP-NodeJS log the interaction with HomeKit.

Setting or Clearing No Response

As mentioned before, the initiative to set or clear must come from HomeKit, so you need to get it to issue a read or write request to the accessory or bridge.
You can trigger a write by changing a characteristic value in Home or another HomeKit app. Triggering a read is very tricky. Sometimes refreshing a view, or changing views in Home or another HomeKit app triggers a read, but other times it doesn't. The only sure way to trigger a read is to force-quit and restart Home.

reachable in the Hue/deCONZ API

Both the Hue and the deCONZ API expose a reachable attribute on resources corresponding to Zigbee devices. This attribute is set on receipt of a message from the device. It is cleared when a few subsequent requests to the device go unanswered. A value false for reachable doesn't mean that the device is currently unreachable, only that it didn't respond the last couple of times that the gateway tried to reach the device.

Power Off

Of course, devices are unreachable when powered off. However, they is no definite way for other devices, including the gateway, to ascertain whether a device is powered off or no response is received for another reason. Not surprisingly, Zigbee doesn't define any message for a device to announce that it's powering down. How would the device send the message without power?

When the firmware of a Zigbee device boots, it broadcasts a Device Announcement message, to announce its presence in the Zigbee network. As the gateway picks this up, it sets reachable. deCONZ keeps track when the last Device Announcement message was received and exposes this as lastboot.

End Devices

Technically, Zigbee end devices are unreachable almost all of the time. They turn off their radio to preserve (battery) power. Messages to an end device are delivered to its parent router, who saves them for 7.5 seconds. Periodically, the end devices wakes up and "polls" its parent, it asks its parent if there's any messages. "Light sleepers" poll their parent router at least every 7.5 seconds, and appear to be responding to requests, albeit with a delay. "Deep sleepers" poll their perent at longer intervals (up to several hours), and appear to be dead. The gateway needs to hold any requests, until the end device is awake. The pending requests are exposed through config.pending.

What Happens When reachable Is false?

The Hue bridge and deCONZ handle a reachable value of false differently:

• The Hue API continues to accept PUT requests, returning OK. Initially, it still sends the corresponding commands to the Zigbee device. After some time, it concludes that the device really is unreachable, stops polling it, and silently drops any requests to it, while still returning OK to the API call;
• The native HomeKit function of the Hue bridge returns an error on read or write, and doesn't send the corresponding Zigbee commands, even when the API still would;
• The deCONZ API returns api error 202 (201 before v2.10) and doesn't send the corresponding Zigbee command(s).

Homebridge Hue

By default, Homebridge Hue exposes reachable as Status Fault. When the API reports a resource with reachable false, Homebridge Hue sets Status Fault. It clears Status Fault when reachable becomes true again. Otherwise, Homebridge Hue does

To change the default behaviour, set wallSwitch, either in config.json for all /lights resources, or using a resource link for individual /lights resources.

Clearing or Setting reachable

I want to change the way Homebridge Hue handles /lights resources with state.reachable set to false.

New Functionality

• Homebridge Hue sets the On (or Active) characteristic to an error value, when the Hue bridge or deCONZ gateway reports state.reachable as false. This is visible in the Homebridge log as:

  [1/16/2021, 4:14:01 PM] [Hue] GLEDOPTO: light reachable changed from true to false
  [1/16/2021, 4:14:01 PM] [Hue] GLEDOPTO: set homekit on from true to No Response
  

• Homebridge Hue will set the On (or Active) characteristic to a normal value, when the Hue bridge or deCONZ gateway reports state.reachable as true. This is visible in the Homebridge log as:

  [1/16/2021, 4:17:56 PM] [Hue] GLEDOPTO: light reachable changed from false to true
  [1/16/2021, 4:18:01 PM] [Hue] GLEDOPTO: set homekit on from No Response to true
  

• Homebridge Hue continues to function normally while state.reachable is false:
  • When a characteristic is changed from HomeKit, it issues a PUT request to the Hue bridge or deCONZ gateway to update the corresponding attribute. The result of that request (success or failure) is passed back to HomeKit;
  • When an attribute is changed on the Hue bridge or deCONZ gateway, Homebridge Hue updates the value of the corresponding HomeKit characteristic.
• Homebridge Hue no longer exposes Status Fault on services corresponding to /lights resources;
• The wallSwitch config.json setting has been removed, as it's no longer needed;
• The wallswitch resource link is no longer supported, as it's no longer needed.

Caveats

• state.reachable does not reflect whether the device is currently reachable, let alone whether it's been powered off.
  • state.reachable is set to false when the bridge/gateway doesn't receive responses for a couple of subsequent Zigbee unicast messages to the device. Depending on the number of devices in your Zigbee network, it can take a good couple of minutes after powering off a light, before state.reachable becomes false;
  • state.reachable is set to true when the bridge/gateway receives a response or notification from the device. As (most) devices send a Device Announcement on boot, state.reachable tends to become true quickly after powering on a light;
• As the error value is pushed to the On (or Active) characteristics, the service will turn off in HomeKit as soon as `state.reachable becomes false.
• HomeKit doesn't change the No Response state when a value is pushed. It's only set or cleared on a response that HomeKit receives when querying or updating the accessory. Sometimes you can force querying by refreshing the view, or switching views in your HomeKit app; other times you need to force quit the app and restart it. As Homebridge Hue continues to handle updated characteristics, these might or might not result in the No Response being set;
• The Hue bridge still accepts requests and sends the corresponding Zigbee commands while state.reachable is false. Of course, On is reset to the error when the light is next polled and state.reachable is still false:

  [1/16/2021, 4:58:14 PM] [Hue] GLEDOPTO: homekit on changed from No Response to true
  [1/16/2021, 4:58:14 PM] [Hue] hue-3: request 830: PUT /lights/27/state {"on":true}
  [1/16/2021, 4:58:14 PM] [Hue] hue-3: request 830: 200 OK
  [1/16/2021, 4:58:15 PM] [Hue] hue-3: request 832: GET /lights
  [1/16/2021, 4:58:15 PM] [Hue] hue-3: request 832: 200 OK
  [1/16/2021, 4:58:15 PM] [Hue] GLEDOPTO: set homekit on from true to No Response
  

• The deCONZ gateway returns an API error when state.reachable is false:

  [1/16/2021, 4:56:59 PM] [Hue] LIDL spot: homekit on changed from No Response to true
  [1/16/2021, 4:56:59 PM] [Hue] pi5: request 819: PUT /lights/10/state {"on":true}
  [1/16/2021, 4:56:59 PM] [Hue] pi5: request 819: 200 OK
  [1/16/2021, 4:56:59 PM] [Hue] pi5: request 819: PUT /lights/10/state {"on":true}
  [1/16/2021, 4:56:59 PM] [Hue] pi5: request 819: api error 202: resource, /lights/10/state, is not modifiable. Device is not reachable.
  

Why This Change?

• Until now, Homebridge Hue never set a No Response state in HomeKit, but used Status Fault instead. Unfortunately, Apple's Home app still doesn't support this standard (!) characteristic. Consequently, you would get no clue in the Home app when a light is (reported) unreachable;
• Most (if not all) other plugins do set the No Response state in HomeKit, and people expect Homebridge Hue to do the same;
• There are still quite a few people who insist on using 20th century wall switches with their smart lights. They want to show the light as off in HomeKit, when (reported) unreachable. Homebridge Hue used the wallSwitch config.json setting and wallswitch resourcelink for this. These functions have been removed, simplifying the code.