Expose bridge/gateway scenes to HomeKit

[ebaauw]

See #158, #373, #472.

Expose Hue bridge/deCONZ gateway scenes as HomeKit switches, cf. Home Assistant.

To prevent creating too many HomeKit accessories, these switches would be created as additional services under the group accessories. This makes sense for scenes linked to a group, like deCONZ gateway scenes and GroupScene Hue bridge scenes. It doesn't make sense for LightScene Hue bridge scenes, which are stored under special group 0 (this is different from group 0 on the deCONZ gateway), but can be recalled using any group. These should probably be exposed under each group.

Adding a scenes key to config.json is easily done. However, as every other Hue app tends to create its own scenes, the number of scenes becomes overwhelming pretty quickly. We probably need a whitelist/blacklist mechanism to control which scenes are exposed. Hue bridge scenes can be added to resourcelinks, but I don't think deCONZ scenes can. We would probably need support for the Hue bridge API's scenes resources in deCONZ, see dresden-elektronik/deconz-rest-plugin#1471.

[robertlandes]

Big thumbs up for this one. I was actually thinking about opening an issue for this topic with almost the same idea behind it for the last 2 weeks. I think this will benefit the whole setup a lot, as I must say creating all those scenes in Homekit is a pain. I setup all scenes both in Homekit AND deCONZ because the configruation options in deCONZ are better especially when combined with switch config like IKEA 5-button remote or Hue dimm switch (i.e. toggle on/off on one button, toggle/cycle through scenes etc).

I think exposing group scenes as additional services is the way to go, otherwise we will hit the 99 items limit very quickly. The question is, will it be just a turn on "switch" or will it reflect the actual activated scene (I don't know if this is possible though).

Regarding the whitelist/blacklist feature, I see several options how to implement it:

• create just a whitelist options for groups which scenes should be exposed. I think this is more suitable (at least for my setup) then having a blacklist feature, as I have a lot of groups and most groups are not configured with scenes. All groups in this array would be exposed via existing homebridge-hue whitelist resourcelink. We would kill two birds with one stone using this method. We don't expose all available groups via using the groups option, but expose only the ones in the whitelist array and enable scenes for all those groups.

• make the whitelist and blacklist resourcelink feature available in general. This would make the config,json more complex (and I understand you are not so happy about introducing complexity to it), however would give a lot more control in the users hand and is for sure easier then using ph to post/put the json for the resourcelink config. Group whitelists would be same as above (not using group setting)

• implement a new feature for white- or blacklisting scenes of groups

I know this might be a little off topic (and I am happy to open a seperate issue if you like), but I think giving user control over whitelist / blacklist as well as outlet resourcelinks via config.json would be awesome. I.e. only light resources especially mentioned under outlet array would be exposed as outlets instead of lights.

[ebaauw]

Big thumbs up for this one. I was actually thinking about opening an issue for this topic with almost the same idea behind it for the last 2 weeks. I think this will benefit the whole setup a lot, as I must say creating all those scenes in Homekit is a pain. I setup all scenes both in Homekit AND deCONZ because the configruation options in deCONZ are better especially when combined with switch config like IKEA 5-button remote or Hue dimm switch (i.e. toggle on/off on one button, toggle/cycle through scenes etc).

In general, deCONZ scenes are to be preferred over HomeKit scenes. Not just from a functional perspective, but also non-functionally: recalling a deCONZ scene results a single Zigbee (broadcast/groupcast) message, whereas recalling a HomeKit scene results in up to three Zigbee (unicast) messages per light in the scene (see #15 (comment)).

I think exposing group scenes as additional services is the way to go, otherwise we will hit the 99 items limit very quickly. The question is, will it be just a turn on "switch" or will it reflect the actual activated scene (I don't know if this is possible though).

That limit has been raised by Apple to 150 (but I still need workarounds not to hit it ;-). I think Zigbee provides a means (per light) to check whether it still reflects the state of the last scene recalled, but the REST API doesn't expose this. I haven't tested how reliable this is, and whether all light types/manufacturers actually support this. You would have to check this for all lights in the scene, to check whether the scene is still active, much like state.all_on and state.any_on for groups. Even if I can get this to work in deCONZ, it won't work for the Hue bridge.

Regarding the whitelist/blacklist feature, I see several options how to implement it:

That's actually a good idea: only expose scenes for whitelisted groups, or for all not-blacklisted groups when groups has been set in config.json.

I know this might be a little off topic (and I am happy to open a seperate issue if you like), but I think giving user control over whitelist / blacklist as well as outlet resourcelinks via config.json would be awesome.

Yes, for people who appreciate the REST API, and understand that it exposes resources, which you whitelist or blacklist. For less tech savvy people (who don't even understand json), this is a nightmare. For me as well - this is a hobby project and I don't want to spend my time on end-user support. I figure if you understand the REST resources, it's a small step to understanding resourcelinks.

Eventually, I want homebridge-hue to provide a user interface that shows the devices (as opposed to resources) connected to the Hue bridge/deCONZ gateway, and allow to check which to expose and, where appropriate, how to expose them. Using dynamic platform accessories (see #4), this would be persisted in ~/.homebridge/accessories/cachedAccessories.

I could provide that UI (somewhat awkwardly) through HomeKit, but you only want a HomeKit administrator to access it. HomeKit seems to provide a facility for that, but no-one seems to know how to use it (see homebridge/homebridge#1389).

So the UI probably needs to be through a separate, password protected, web server (like homebridge-config-ui-x). Unfortunately, the node modules they use to render the web UI are written in C/C++, causing a load of installation issues on containers, virtual machines, and exotic platforms. That's a no-go for me as well.

So I need to find a native JavaScript module to generate a web UI, or write my own. Not my favourite past time; I started with HomeKit in the first place, so I wouldn't have to create my own UI. Anyway, I would need to refactor homebridge-hue first, to switch to the dynamic platform accessory model.

[robertlandes]

That's actually a good idea! Exposing scenes from groups which are whitelisted or expose scenes from all groups except those which are blacklisted when using groups = true in config.json.

The only thing which might be tricky (at least when you make white- and blacklist configurable through config.json) is that you would need to keep track of the ID of the resourcelink and worst case scenario to keep track of additional items already included (via ph for example) and merge those lists from resourcelink and config.json before updating resourcelink.

So maybe as a first step implement exposing scenes as discussed above and just make it configurable through REST API / ph keeping it a feature for advanced users and if things are running smoothly and only if there is an existing demand, make it configurable through config.json?

Regarding regarding your "hobby project"... You do an amazing job on this (and the Sonos) project.. Thanks again!

[ebaauw]

v0.11.32 supports deCONZ scenes:

• Set "scenes": true in config.json to expose the scenes for any exposed group.
• As always, use the "groups" and "group0" config.json settings and/or white- or blacklists to specify which groups are exposed.
• Each scene is exposed as an additional Switch service under the group accessory, with a single On characteristic.
• When setting On true, the scene is recalled.
• The switch resets to On false after a second, as there's no way (at least through the API) to detect whether the scene is still active.
• Note that Brightness, Color Temperature, and Color for a group are write-only, unline On and AnyOn. They are used for changing the states of the lights in the group. They do not reflect the actual state of the lights in the group (again, this is a limitation of the API, group state vs group action). In particular, they do not reflect the state when a scene is recalled.

Please let me know if this works for you.

Hue bridge scenes are a bit more elaborate, as scenes are stored as common /scenes resources, but can be recalled from any group. I'll have to expose the scenes under each exposed group.

[ebaauw]

Actually, this doesn't work: too many tiles in Apple's Home app and Siri wants to recall all scenes when setting a room on. So I better expose the scenes as custom Resource services with the custom Enabled characteristic (cf. schedules and rules). To recall a scene from Home or Siri, you will have to create a HomeKit scene using Eve (setting Enabled of the corresponding deCONZ scene), but this gives you full control what scenes to display in Home.

EDIT in v0.11.34.

[anjos]

I'd like to test this feature, my config looks like this:

>> cat /var/lib/homebridge/config.json
{
    "bridge": {
        "name": "Phoscon gateway",
        "username": "---------",
        "port": 51826,
        "pin": "--------"
    },
    "description": "Phoscon deCONZ -> HomeKit Bridge",
    "platforms": [
      {
        "platform": "Hue",
        "host": "127.0.0.1",
        "users": {
          "---------------------": "--------------"
        },
        "rooms": true,
        "sensors": true,
        "scenes": true,
        "excludeSensorTypes": ["Daylight", "CLIP", "Geofence", "ZLLSwitch", "ZGPSwitch"],
        "lights": true,
        "waitTimeUpdate": 100
      }
    ]
}

But I cannot see any resource available for the scenes I have setup. For example:

>> ph -H 127.0.0.1 get /groups/2/scenes
{
  "1": {
    "lights": [
      "3",
      "4"
    ],
    "name": "Reading",
    "transitiontime": 10
  },
  "2": {
    "lights": [
      "3",
      "4"
    ],
    "name": "Sleeping",
    "transitiontime": 10
  }
}

Doesn't show up in Eve or Home apps. Did I miss something?

[ebaauw]

Yes, you need to expose groups, see above:

• Set "scenes": true in config.json to expose the scenes for any exposed group.
• As always, use the "groups" and "group0" config.json settings and/or white- or blacklists to specify which groups are exposed.

[anjos]

I confirm that this works for me. It simplified most HomeKit scenes I had to a single command, which is great.

A curiosity: is there a way to emulate the "All off" functionality from deconz's web interface through this mechanism? I did not find any way other than setting up an equivalent scene where all lights are in.

[ebaauw]

Expose /groups/0 by setting group0 in config.json.

[anjos]

Yes, I did that. Group 0 is exposed. If I set a new HomeKit scene (through Eve), in which the power on group 0 is set to OFF, then this has no effect on the lights. So, effectively, I have no way to turn off all lights at once.

The only way I found so far was to create a scene inside a group using all lights, that sets everything to OFF. If I do that, I can call this scene through Eve and all lights will turn off.

The annoying issue with this solution is that if I add a new light, I have to remember to update this scene as well, so it turns off as expected.

[ebaauw]

deCONZ v2.05.66? It has some issues with /groups/0, see dresden-elektronik/deconz-rest-plugin#1612.

[realdannys]

Hi Eric,

Over from the linked post above.

Yes for me the idea of a momentary switch to trigger a Hue Bridge scene would be best. I wouldn’t really want them to appear as scenes in Homekit, just the ability to turn them on is enough.

My usage case would be something like

1. Create 30 second fading scene in Hue Lights app and upload to Hue Bridge
2. Expose that specific scene as a switch in Homebridge config via your plugin.

[ebaauw]

My issue is how to configure what scenes to expose and how. Hue bridge scenes are defined independently from groups (unlike deCONZ scenes); any group could be used to recall the scene, changing only the lights that are both in the scene as well as in the group. So, by default, I would expose every scene under every (exposed) group.

Note that Siri gets hopelessly confused when exposing scenes as HomeKit switches - she’ll activate all scenes when you ask her to turn on a room. That’s why I use a custom characteristic to expose them (same as for enabling/disabling schedules). In Eve these look just like switches, but in Home, they’re not visible. The workaround for this is to create (in Eve) a HomeKit scene with the custom switch, which is visible In and can be activated from Home.

Note that with homebridge-hue, you could also create a fading scene in HomeKit. It exposes transitiontime under the bridge accessory. Again, Home doesn’t support this, but if you create the scene in Eve, Home will happily activate it.

[realdannys]

Your last part is probably my easiest way to sort this.
I haven’t added Homebridge-Hue yet but I assume I don’t need to remove natively added Hue stuff to Homekit? I can just expose the bridge and create scenes with transitions from there?

Does it support blink and colour loop too?

[ebaauw]

You can keep the lights exposed natively, but you would have to expose lights or groups thru homebridge-hue as well; the transitiontime needs to be added to the Hue API call to change the light or group.

I don’t support colorloop, but that would be an easy enhancement. I blink the lights when you identify them in HomeKit, but I’ve never seen a HomeKit app that lets you add Identify to a scene.

[ebaauw]

My issue is how to configure what scenes to expose and how.

I think I cracked it. The Hue app creates scenes for a particular room. This room (or rather: the group for this room) is encoded in the appdata object in the /scenes resource. This allows me to expose the scene only for that group. So, in order to expose Hue bridge scenes, you need:

• To set "scenes": true in config.json;
• To expose the room group, either by whitelisting it, or by setting "groups": true and "rooms": true in config.json;
• To create the scene using the Hue app, or create the scene with appdata similar to the Hue app: { version: 1, data: "xxxxx_rrr_ddd" }, where the rr refers to the id of the room group.

[ebaauw]

v0.11.48 supports Hue bridge scenes and colorloop.

[Joebinator]

Hi there!

How do i get to see my hue scenes after i updated to v0.11.48.

I have set: "scenes": true, "groups": true and "rooms": true in config.json;

[ebaauw]

They're exposed under the accessory for the (room) group. They use a custom characteristic (or Siri would activate all scenes when you ask her to turn on a room), so you need to use a decent HomeKit app, like Eve, instead of Apple's Home. To recall the Hue bridge scenes from Siri or Home, create a HomeKit scene in Eve, with the custom characteristic to recall the bridge scene.