Cloud API about to change by the looks of it

[pmcgaley]

In case you didn't get/see this from EPH, presumably relevant...

Please note that a major release of EMBER will be available to download on January 4th. It is important that you update the EMBER App to this latest version as the old version of the App will no longer control your heating system in the coming weeks.
Improvements include:
Optimisation of APP to ensure smoother communication
Known bugs fixed

[ttroy50]

thanks for the heads up. I'll see if I can get some time to investigate the new API and update

[UtzR]

It seems the protocol has slightly changed and that has broken things.

in get_home() it drops out with error 32:
{'timestamp': 1612988335736, 'status': 32, 'data': None, 'message': 'WiFi device is not online, please check and try again.'}

COMMENT
When I have the phone app running and run the python API I do not get this error but it fails then later in another call.

It seems that there is an attempt to exclude anything that is not a phone app from calling the API.
COMMENT

I cannot replicate the above behaviour anymore. It now drops out with this error 32 regardless if I interact with the phone app at the same time.

[ttroy50]

Thanks. I've just started to look at the new API and will hopefully have an update soon.

[ttroy50]

See this issue in home assistant for some more discussion recent API issues

home-assistant/core#46223

[UtzR]

In

def get_home(self, gateway_id=None):

the url, headers and data submitted in the request are:

https://eu-https.topband-cloud.com/ember-back/zones/polling 
{'Authorization': 'XXXXXXXXXXXXXXXXXXXXX', 'Accept': 'application/json', 'Content-Type': 'application/json'} 
{'gateWayId': 'XXXXXXXXX'}

the response is
<Response [200]>

And response.json() looks like this:
{'status': 32, 'message': 'WiFi device is not online, please check and try again.', 'timestamp': 1613218949350, 'data': None}

And it fails with:
Traceback (most recent call last):
File "openhabEMBER.py", line 122, in
sys.exit(main())
File "openhabEMBER.py", line 38, in main
print(json.dumps(t.get_zones(), indent=4, sort_keys=True))
File "/opt/openhab-ember/pyephember-master/pyephember/pyephember.py", line 292, in get_zones
home_data = self.get_home()
File "/opt/openhab-ember/pyephember-master/pyephember/pyephember.py", line 284, in get_home
"Error getting zones from home: {}".format(status))
RuntimeError: Error getting zones from home: 32

[ttroy50]

I've pushed https://github.com/ttroy50/pyephember/blob/update_api_docs/API.md which has some of the work from my documentation of the new API. unfortunately this is from a couple of weeks ago and I haven't had a chance to expand on it yet. I'm getting very little time to look into this at the moment.

[spoonlyorange]

Hi @ttroy50, I'd like to help out and contribute to this, but I'm having difficulty capturing the traffic. How are you doing it? I used polarproxy, but it's not able to establish a connection to the mqtt server, so I'm not getting a pcap output for this traffic. It's probably a local issue though.

I was able to modify mitmproxy-mqtt-script to give me hex output, but I see your output looks like you're opening it in WireShark.

I did find that they might be using EnOS (https://github.com/EnvisionIot) as their IoT manager. It uses port 18883 for MQTTS, when the standard is 8883. So once we can figure out the rest of the parameters used by Ember, we might be able to use that.

[DarkSlice1]

Curious if this still works
Been reading the Api doc that @ttroy50 updated, i can get a token but no other call passes
Always get
{
"message": "Not logged in or session timeout.",
"status": 9,
"timestamp": 1638530699855
}

Im using the following auth format for auth "Authorization: Bearer XXXX" -> where XXXX = "data.token" from login
It might be related to the reportToken step but ive no idea what "really_long_phone_token" is representing (i've tried unique UUID 4 without much luck)

"phoneToken": "really_long_phone_token"

I've also noticed its HTTP 2.0 - something postman doesn't yet support, is the a hard requirement?

Curious how others are getting on.

[mcgizzle]

@DarkSlice1 you are passing the auth token slightly incorrectly, and the error message is very opaque.

It's just

Authorization: token

You just need to drop the bearer, since they are using a custom auth mechanism.

[DarkSlice1]

ah perfect, thank you 👍

[DarkSlice1]

Point index 10, if value = 2, then the boiler is active and burning oil

[rupertleveneucd]

The code in pull request #13 is working for me with the current API.

[UtzR]

The code in pull request 13 works for me (I was looking for get_zone_status() which is now is_zone_boiler_on() but it all seems to work. Thanks.

[ttroy50]

If anyone else here could test the code from #13 and let us know if it works it would be very helpful towards getting a new release out.

Thanks to @rupertleveneucd for doing the change.

[dashford]

Hi @ttroy50, I've tested it with the latest version from master and all works great. Would be super to get a release out and into home assistant - anything I could do there to help with that?

[roberty99]

I've tested latest version from master and working for me well. How can we go about getting ephember home assistant updated with this method.

[roberty99]

@ttroy50 I think all that is needed is to bump this master version on pypi.org. The EPHcontrol in home assistant already pull this as a requirement in it's manifest file. It's currently pulling version 3.9. So we can clone that to a custom_component and test there.

[UtzR]

Is it not version 0.3.1? The manifest looks for 'pyephember 0.3.1'? Where do you get the 3.9 version number (Maybe I look at the wrong place). Is there a way to place the correct library with a custom_component to bypass the update on pypi.org? I am happy to test.

[roberty99]

You are correct 0.3.1 apologies. I have my own custom component but the Requirements paramater in the manifest.json no longer allow you to specify a GIT. Has to come from pypi.https://www.google.com/url?sa=t&source=web&rct=j&url=https://developers.home-assistant.io/docs/creating_component_code_review/&ved=2ahUKEwjhjPbOwZD8AhVTZ8AKHUxvDbYQFnoECA0QAQ&usg=AOvVaw0yDfe2nFRX8nPONLZZQ3wc

[UtzR]

Not yet familiar with home assistant, currently try to switch from openhab to that. Is it not possible to have the library directly within the custom_component and not use the requirements mechanism to retrieve that from pypi.org. Ideally an update in pypi.org is better but as a solution in the meantime.

[roberty99]

ok I was able to do a pip install in the home assistant container directly. The method from here https://developers.home-assistant.io/docs/creating_integration_manifest/

Then the custom component loaded but threw some errors around a is_hot_water so I removed all entries around that.
Reloaded and now I can see and add my zones in HA. Very nice.
Most of the calls don't work but its a good step forward !

[UtzR]

Some of the api has changed. For example get_zone_status() which is now is_zone_boiler_on() and the is_hot_water does not exist anymore; so some adjustment might be needed to get everything working.

Not sure how you managed to install this, I need to spend some time to figure out how to do the pip install on a docker container I am running for this ...

[roberty99]

add the standard install to you config file.
climate:

• platform: ephember
  username: '@.'
  password: '**'

Create a folder under custom_componets called ephember. Needs to be the same name to override the standard component.
Copy in 3 files from the main source component. https://github.com/home-assistant/core/tree/dev/homeassistant/components/ephember

Remove the version from the requirement in manifest.json

Get to the homeassistant container console. In my case through portioner as I have HASSOS

Git Clone https://github.com/ttroy50/pyephember.git
Pip install -e ./pyephember

Restart HA and check the log file of errors.
Remove lines from climate.py referring stuff that doesn't exist anymore.
Restart once more and the integration is loaded. can add my zones and most of the gets work.

Would need someone who understands the API to adjust for the changes since the last time this was working was 2 years ago.

[UtzR]

It took me a while. I have a docker container, so no ssh and no shell; needed to learn you can use 'docker exec' to execute commands in the container and then did the above. Run into some issues.

I did not understand what you mean exactly by 'Remove the version from the requirement in manifest.json'. So instead I set the version to 0.4.0 which is the new installed pyephember.

I restarted home assistant and got these two errors:

Platform error: climate - Requirements for ephember not found: ['pyephember==0.3.1'].
22:31:13 – (ERROR) config.py

Unable to install package pyephember==0.3.1: ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/.local' Check the permissions. [notice] A new release of pip available: 22.3 -> 22.3.1 [notice] To update, run: pip install --upgrade pip
22:31:13 – (ERROR) util/package.py - message first occurred at 22:30:33 and shows up 3 times

So it seems it wants to install the old version again, how do you prevent this?

---> Found out the manifest.json needs a version number otherwise home assistant takes the original version and not the new one in custom_components

[roberty99]

I think your custom component isn't getting picked up. Hence the default component is complaining. Did you add a version param to the manifest file ?

…

On Fri 23 Dec 2022, 22:42 UtzR, ***@***.***> wrote: It took me a while. I have a docker container, so no ssh and no shell; needed to learn you can use 'docker exec' to execute commands in the container and then did the above. Run into some issues. I did not understand what you mean exactly by 'Remove the version from the requirement in manifest.json'. So instead I set the version to 0.4.0 which is the new installed pyephember. I restarted home assistant and got these two errors: Platform error: climate - Requirements for ephember not found: ['pyephember==0.3.1']. 22:31:13 – (ERROR) config.py Unable to install package pyephember==0.3.1: ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/.local' Check the permissions. [notice] A new release of pip available: 22.3 -> 22.3.1 [notice] To update, run: pip install --upgrade pip 22:31:13 – (ERROR) util/package.py - message first occurred at 22:30:33 and shows up 3 times So it seems it wants to install the old version again, how do you prevent this? — Reply to this email directly, view it on GitHub <#11 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALVJ7EO3W7OLDP64ECAKPCLWOYTGVANCNFSM4VOKRRFA> . You are receiving this because you commented.Message ID: ***@***.***>

[roberty99]

OK some more time to play with it today so I have fixed up the boost (aux), set_temperature and set_mode functions.
I had to remove the Hot Water zone detection from the old methods as I'm not so good to at coding to figure out how that works.I think it is device_type, Basically if it is a hot water zone the temperature set isn't supposed to be set. Be careful with that hopefully someone with more knowledge can make it work like it use to. I set the max temp to be 50 so I can control the hot water.

Original code: https://github.com/home-assistant/core/blob/dev/homeassistant/components/ephember/climate.py

"""Support for the EPH Controls Ember themostats."""
from __future__ import annotations

from datetime import timedelta
import logging
from typing import Any

from pyephember.pyephember import (
    EphEmber,
    ZoneMode,
    zone_current_temperature,
    zone_is_active,
    zone_is_boost_active,
    zone_boost_hours,
    zone_boost_timestamp,
    zone_advance_active,
    boiler_state,
    zone_is_scheduled_on,
    zone_mode,
    zone_name,
    zone_temperature,
    zone_target_temperature,
    zone_boost_temperature,
    zone_current_temperature,
    zone_pointdata_value,
)
import voluptuous as vol

from homeassistant.components.climate import (
    PLATFORM_SCHEMA,
    ClimateEntity,
    ClimateEntityFeature,
    HVACAction,
    HVACMode,
)
from homeassistant.const import (
    ATTR_TEMPERATURE,
    CONF_PASSWORD,
    CONF_USERNAME,
    UnitOfTemperature,
)
from homeassistant.core import HomeAssistant
import homeassistant.helpers.config_validation as cv
from homeassistant.helpers.entity_platform import AddEntitiesCallback
from homeassistant.helpers.typing import ConfigType, DiscoveryInfoType

_LOGGER = logging.getLogger(__name__)

# Return cached results if last scan was less then this time ago
SCAN_INTERVAL = timedelta(seconds=10)

OPERATION_LIST = [HVACMode.HEAT_COOL, HVACMode.HEAT, HVACMode.OFF]

PLATFORM_SCHEMA = PLATFORM_SCHEMA.extend(
    {vol.Required(CONF_USERNAME): cv.string, vol.Required(CONF_PASSWORD): cv.string}
)

EPH_TO_HA_STATE = {
    "AUTO": HVACMode.HEAT_COOL,
    "ON": HVACMode.HEAT,
    "OFF": HVACMode.OFF,
}

HA_STATE_TO_EPH = {value: key for key, value in EPH_TO_HA_STATE.items()}


def setup_platform(
    hass: HomeAssistant,
    config: ConfigType,
    add_entities: AddEntitiesCallback,
    discovery_info: DiscoveryInfoType | None = None,
) -> None:
    """Set up the ephember thermostat."""
    username = config.get(CONF_USERNAME)
    password = config.get(CONF_PASSWORD)

    try:
        ember = EphEmber(username, password)
        zones = ember.get_zones()
        for zone in zones:
            add_entities([EphEmberThermostat(ember, zone)])
    except RuntimeError:
        _LOGGER.error("Cannot connect to EphEmber")
        return

    return


class EphEmberThermostat(ClimateEntity):
    """Representation of a EphEmber thermostat."""

    _attr_hvac_modes = OPERATION_LIST
    _attr_temperature_unit = UnitOfTemperature.CELSIUS

    def __init__(self, ember, zone):
        """Initialize the thermostat."""
        self._ember = ember
        self._zone_name = zone_name(zone)
        self._zone = zone

        self._attr_name = self._zone_name

        self._attr_supported_features = (
            ClimateEntityFeature.TARGET_TEMPERATURE | ClimateEntityFeature.AUX_HEAT
        )
        self._attr_target_temperature_step = 0.5

    @property
    def current_temperature(self):
        """Return the current temperature."""
        return zone_current_temperature(self._zone)

    @property
    def target_temperature(self):
        """Return the temperature we try to reach."""
        return zone_target_temperature(self._zone)

    @property
    def hvac_action(self) -> HVACAction:
        """Return current HVAC action."""
        if zone_is_active(self._zone):
            return HVACAction.HEATING

        return HVACAction.IDLE

    @property
    def hvac_mode(self) -> HVACMode:
        """Return current operation ie. heat, cool, idle."""
        mode = zone_mode(self._zone)
        return self.map_mode_eph_hass(mode)

    def set_hvac_mode(self, hvac_mode: HVACMode) -> None:
        """Set the operation mode."""
        mode = self.map_mode_hass_eph(hvac_mode)
        if mode is not None:
            self._ember.set_zone_mode(self._zone_name, mode)
        else:
            _LOGGER.error("Invalid operation mode provided %s", hvac_mode)

    @property
    def is_aux_heat(self):
        """Return true if aux heater."""

        return zone_is_boost_active(self._zone)

    def turn_aux_heat_on(self) -> None:
        """Turn auxiliary heater on."""
        self._ember.activate_zone_boost(
            self._zone_name, zone_target_temperature(self._zone)
        )

    def turn_aux_heat_off(self) -> None:
        """Turn auxiliary heater off."""
        self._ember.deactivate_zone_boost(self._zone_name)

    def set_temperature(self, **kwargs: Any) -> None:
        """Set new target temperature."""
        if (temperature := kwargs.get(ATTR_TEMPERATURE)) is None:
            return


        if temperature == self.target_temperature:
            return

        if temperature > self.max_temp or temperature < self.min_temp:
            return

        self._ember.set_zone_target_temperature(self._zone_name, temperature)

    @property
    def min_temp(self):
        """Return the minimum temperature."""
        # Hot water temp doesn't support being changed

        return 5.0

    @property
    def max_temp(self):
        """Return the maximum temperature."""

        return 50.0

    def update(self) -> None:
        """Get the latest data."""
        self._zone = self._ember.get_zone(self._zone_name)

    @staticmethod
    def map_mode_hass_eph(operation_mode):
        """Map from Home Assistant mode to eph mode."""
        return getattr(ZoneMode, HA_STATE_TO_EPH.get(operation_mode), None)

    @staticmethod
    def map_mode_eph_hass(operation_mode):
        """Map from eph mode to Home Assistant mode."""
        return EPH_TO_HA_STATE.get(operation_mode.name, HVACMode.HEAT_COOL)

[UtzR]

just tested your code and that works. Thanks. I will experiment with it for a bit more ...

[roberty99]

@ttroy50 could you bump the Master 4.0 version to the latest on PyPi ? https://pypi.org/project/pyephember/

[ajurna]

I have device type 4 on my immersion sensor and 2 on my normal rooms if that's any help.
i have supplied my version of the code which keeps the hot water code but disables the detection. if we know the device type for that it could be simply updated.
i added support for immersions which have a temp limit of 90. (at least on the controller)

"""Support for the EPH Controls Ember themostats."""
from __future__ import annotations

from datetime import timedelta
import logging
from typing import Any

from pyephember.pyephember import (
    EphEmber,
    ZoneMode,
    zone_current_temperature,
    zone_is_active,
    zone_is_boost_active,
    zone_mode,
    zone_name,
    zone_target_temperature
)
import voluptuous as vol

from homeassistant.components.climate import (
    PLATFORM_SCHEMA,
    ClimateEntity,
    ClimateEntityFeature,
    HVACAction,
    HVACMode,
)
from homeassistant.const import (
    ATTR_TEMPERATURE,
    CONF_PASSWORD,
    CONF_USERNAME,
    UnitOfTemperature,
)
from homeassistant.core import HomeAssistant
import homeassistant.helpers.config_validation as cv
from homeassistant.helpers.entity_platform import AddEntitiesCallback
from homeassistant.helpers.typing import ConfigType, DiscoveryInfoType

_LOGGER = logging.getLogger(__name__)

# Return cached results if last scan was less then this time ago
SCAN_INTERVAL = timedelta(seconds=120)

OPERATION_LIST = [HVACMode.HEAT_COOL, HVACMode.HEAT, HVACMode.OFF]

PLATFORM_SCHEMA = PLATFORM_SCHEMA.extend(
    {vol.Required(CONF_USERNAME): cv.string, vol.Required(CONF_PASSWORD): cv.string}
)

EPH_TO_HA_STATE = {
    "AUTO": HVACMode.HEAT_COOL,
    "ON": HVACMode.HEAT,
    "OFF": HVACMode.OFF,
}

HA_STATE_TO_EPH = {value: key for key, value in EPH_TO_HA_STATE.items()}


def setup_platform(
    hass: HomeAssistant,
    config: ConfigType,
    add_entities: AddEntitiesCallback,
    discovery_info: DiscoveryInfoType | None = None,
) -> None:
    """Set up the ephember thermostat."""
    username = config.get(CONF_USERNAME)
    password = config.get(CONF_PASSWORD)

    try:
        ember = EphEmber(username, password)
        
        zones = ember.get_zones()
        for zone in zones:
            add_entities([EphEmberThermostat(ember, zone)])
    except RuntimeError as e:
        _LOGGER.error("Cannot connect to EphEmber")
        return

    return


class EphEmberThermostat(ClimateEntity):
    """Representation of a EphEmber thermostat."""

    _attr_hvac_modes = OPERATION_LIST
    _attr_temperature_unit = UnitOfTemperature.CELSIUS

    def __init__(self, ember, zone):
        """Initialize the thermostat."""
        self._ember = ember
        self._zone_name = zone_name(zone)
        self._zone = zone
        self._hot_water = False
        # self._immersion = False
        self._immersion = zone['deviceType'] == 4
        
        self._attr_name = self._zone_name

        self._attr_supported_features = (
            ClimateEntityFeature.TARGET_TEMPERATURE | ClimateEntityFeature.AUX_HEAT
        )
        self._attr_target_temperature_step = 0.5
        if self._hot_water:
            self._attr_supported_features = ClimateEntityFeature.AUX_HEAT
            self._attr_target_temperature_step = None

    @property
    def current_temperature(self):
        """Return the current temperature."""
        return zone_current_temperature(self._zone)

    @property
    def target_temperature(self):
        """Return the temperature we try to reach."""
        return zone_target_temperature(self._zone)

    @property
    def hvac_action(self) -> HVACAction:
        """Return current HVAC action."""
        if zone_is_active(self._zone):
            return HVACAction.HEATING

        return HVACAction.IDLE

    @property
    def hvac_mode(self) -> HVACMode:
        """Return current operation ie. heat, cool, idle."""
        mode = zone_mode(self._zone)
        return self.map_mode_eph_hass(mode)

    def set_hvac_mode(self, hvac_mode: HVACMode) -> None:
        """Set the operation mode."""
        mode = self.map_mode_hass_eph(hvac_mode)
        if mode is not None:
            self._ember.set_mode_by_name(self._zone_name, mode)
        else:
            _LOGGER.error("Invalid operation mode provided %s", hvac_mode)

    @property
    def is_aux_heat(self):
        """Return true if aux heater."""

        return zone_is_boost_active(self._zone)

    def turn_aux_heat_on(self) -> None:
        """Turn auxiliary heater on."""
        self._ember.activate_zone_boost(
            self._zone_name, zone_target_temperature(self._zone)
        )

    def turn_aux_heat_off(self) -> None:
        """Turn auxiliary heater off."""
        self._ember.deactivate_zone_boost(self._zone_name)

    def set_temperature(self, **kwargs: Any) -> None:
        """Set new target temperature."""
        if (temperature := kwargs.get(ATTR_TEMPERATURE)) is None:
            return

        if self._hot_water:
            return

        if temperature == self.target_temperature:
            return

        if temperature > self.max_temp or temperature < self.min_temp:
            return

        self._ember.set_zone_target_temperature(self._zone_name, temperature)

    @property
    def min_temp(self):
        """Return the minimum temperature."""
        # Hot water temp doesn't support being changed
        if self._hot_water:
            return zone_target_temperature(self._zone)

        return 5.0

    @property
    def max_temp(self):
        """Return the maximum temperature."""
        if self._hot_water:
            return zone_target_temperature(self._zone)

        if self._immersion:
            return 90.0

        return 35.0

    def update(self) -> None:
        """Get the latest data."""
        self._zone = self._ember.get_zone(self._zone_name)

    @staticmethod
    def map_mode_hass_eph(operation_mode):
        """Map from Home Assistant mode to eph mode."""
        return getattr(ZoneMode, HA_STATE_TO_EPH.get(operation_mode), None)

    @staticmethod
    def map_mode_eph_hass(operation_mode):
        """Map from eph mode to Home Assistant mode."""
        return EPH_TO_HA_STATE.get(operation_mode.name, HVACMode.HEAT_COOL)

[ttroy50]

Sorry for not getting back. I’ve started to add some of you as collaborators on the project to allow you to make updates

[spoonlyorange]

Sorry for not getting back. I’ve started to add some of you as collaborators on the project to allow you to make updates

Thanks @ttroy50. Can you please also give someone maintainer access to the pypi project so that it can be updated?

[roberty99]

I bumped the Pypi to the 4.0 version now after I got maintainer access last night

[roberty99]

So would just need someone to fix up climate.py on the offical home assistant component and we are good to go. I am currently using my own Custom Component. Don't feel confident enough to suggest all the changes. https://github.com/home-assistant/core/blob/dev/homeassistant/components/ephember/climate.py

[photogaff]

Hi guys, great work on the API - I was almost certain things were working recently however I just started a fresh copy and I'm getting an error thrown at the following:

e.get_zones();
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.6/site-packages/pyephember/pyephember.py", line 676, in get_zones
home_data = self.get_home()
File "/usr/local/lib/python3.6/site-packages/pyephember/pyephember.py", line 650, in get_home
data={"gateWayId": gateway_id}
File "/usr/local/lib/python3.6/site-packages/pyephember/pyephember.py", line 413, in _http
"{} response code".format(response.status_code)
RuntimeError: 500 response code

Any idea what might be going on?

def get_home(self, gateway_id=None):
"""
Get the data about a home (API call: homesVT/zoneProgram).
If no gateway_id is passed, the first gateway found is used.
"""
if gateway_id is None:
if not self._homes:
self._homes = self.list_homes()
gateway_id = self._get_first_gateway_id()

    response = self._http(
        "homesVT/zoneProgram", send_token=True,
        data={"gateWayId": gateway_id}
    )

...............