Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

irk_enrollment component for esphome - build issues, license etc #4

Open
agittins opened this issue Mar 21, 2024 · 15 comments
Open

irk_enrollment component for esphome - build issues, license etc #4

agittins opened this issue Mar 21, 2024 · 15 comments

Comments

@agittins
Copy link

Hi David, this is amazing stuff!

I am particularly interested in seeing if your irk_enrollment component can be used to feed IRKs to the Private BLE Device integration that is now in HA core.

I am having trouble getting it to build.

I tried using the component block in my yaml, but it seems that esphome now requires the parent directory to be named components for it to consider importing it.

After cloning your repo locally and creating a symlink, it complained about not finding the text_sensor.h, adding an empty text_sensor: block to my yaml fixed that (the build setup seemed to not realise it needed to copy it in).

Now I am getting:

src/esphome/components/irk_enrollment/irk_enrollment.cpp: In member function 'virtual void esphome::irk_enrollment::IrkEnrollmentComponent::setup()':
src/esphome/components/irk_enrollment/irk_enrollment.cpp:39:84: error: no matching function for call to 'esphome::esp32_ble_server::BLEServer::create_service(int, bool)'
   this->service_ = esp32_ble_server::global_ble_server->create_service(0x1812, true);
                                                                                    ^
In file included from src/esphome/components/irk_enrollment/irk_enrollment.h:14,
                 from src/esphome/components/irk_enrollment/irk_enrollment.cpp:1:
src/esphome/components/esp32_ble_server/ble_server.h:54:8: note: candidate: 'void esphome::esp32_ble_server::BLEServer::create_service(esphome::esp32_ble::ESPBTUUID, bool, uint16_t, uint8_t)'
   void create_service(ESPBTUUID uuid, bool advertise = false, uint16_t num_handles = 15, uint8_t inst_id = 0);
        ^~~~~~~~~~~~~~
src/esphome/components/esp32_ble_server/ble_server.h:54:8: note:   no known conversion for argument 1 from 'int' to 'esphome::esp32_ble::ESPBTUUID'
Compiling .pioenvs/ttgotest/src/esphome/components/light/esp_color_correction.o
In file included from src/esphome/components/sensor/sensor.h:3,
                 from src/esphome/core/application.h:16,
                 from src/esphome/components/irk_enrollment/irk_enrollment.cpp:3:
src/esphome/components/irk_enrollment/irk_enrollment.cpp: In member function 'virtual void esphome::irk_enrollment::IrkEnrollmentComponent::loop()':
src/esphome/components/irk_enrollment/irk_enrollment.cpp:71:17: error: format '%s' expects argument of type 'char*', but argument 5 has type 'std::__cxx11::basic_string<char>' [-Werror=format=]
   ESP_LOGI(TAG, "      irk: %s", irkStr);
                 ^~~~~~~~~~~~~~~
src/esphome/core/log.h:72:36: note: in definition of macro 'ESPHOME_LOG_FORMAT'
 #define ESPHOME_LOG_FORMAT(format) format
                                    ^~~~~~
src/esphome/core/log.h:151:28: note: in expansion of macro 'esph_log_i'
 #define ESP_LOGI(tag, ...) esph_log_i(tag, __VA_ARGS__)
                            ^~~~~~~~~~
src/esphome/components/irk_enrollment/irk_enrollment.cpp:71:3: note: in expansion of macro 'ESP_LOGI'
   ESP_LOGI(TAG, "      irk: %s", irkStr);
   ^~~~~~~~
Compiling .pioenvs/ttgotest/src/esphome/components/light/esp_hsv_color.o
cc1plus: some warnings being treated as errors
*** [.pioenvs/ttgotest/src/esphome/components/irk_enrollment/irk_enrollment.o] Error 1

I suspect that it's just that some things have changed in esphome since you wrote the component.

Is this component something you'd be interested in updating for the current esphome (assuming it's not just something bone-headed I'm doing on my end, which is entirely possible!), or perhaps submitting to esphome for inclusion in core? I couldn't find a license statement so thought I'd check in with you to see what you are up for before getting in too deep myself :-)

There's a feature request in the esphome tracker that I'll link to this issue as well.

@dgrnbrg
Copy link
Owner

dgrnbrg commented Apr 5, 2024

Hello! Yes, I would be happy to have this work with the HA in-built functionality.

I've pushed some changes to get it to build with the latest ESPHome. I'm not going to be able to test it for a while, but you could try and give it a spin, and it hopefully will work. I don't know to what extent the ble server code has changed in esphome, but hopefully it's just surface level, and this will work now.

Alternatively, you can checkout the prior commit, and use ESPHome version 2023.9, which definitely will work.

@luzik
Copy link

luzik commented Apr 8, 2024

On Arduino esp32 it is compiling but can't find any devices on my iPhone. On espidf with esp32s3 got this:

In file included from src/esphome/components/sensor/sensor.h:3,
                 from src/esphome/core/application.h:16,
                 from src/esphome/components/irk_enrollment/irk_enrollment.cpp:3:
src/esphome/components/irk_enrollment/irk_enrollment.cpp: In member function 'virtual void esphome::irk_enrollment::IrkEnrollmentComponent::loop()':
src/esphome/components/irk_enrollment/irk_enrollment.cpp:73:17: error: format '%s' expects argument of type 'char*', but argument 5 has type 'std::__cxx11::basic_string<char>' [-Werror=format=]
   ESP_LOGI(TAG, "      irk: %s", irkStr);
                 ^~~~~~~~~~~~~~~
src/esphome/core/log.h:72:36: note: in definition of macro 'ESPHOME_LOG_FORMAT'
 #define ESPHOME_LOG_FORMAT(format) format
                                    ^~~~~~
src/esphome/core/log.h:151:28: note: in expansion of macro 'esph_log_i'
 #define ESP_LOGI(tag, ...) esph_log_i(tag, __VA_ARGS__)
                            ^~~~~~~~~~
src/esphome/components/irk_enrollment/irk_enrollment.cpp:73:3: note: in expansion of macro 'ESP_LOGI'
   ESP_LOGI(TAG, "      irk: %s", irkStr);
   ^~~~~~~~
Compiling .pioenvs/garaz-esp/src/esphome/components/sensor/sensor.o
cc1plus: some warnings being treated as errors
Compiling .pioenvs/garaz-esp/src/esphome/components/socket/bsd_sockets_impl.o
*** [.pioenvs/garaz-esp/src/esphome/components/irk_enrollment/irk_enrollment.o] Error 1
========================= [FAILED] Took 14.28 seconds =========================

@luzik
Copy link

luzik commented Apr 8, 2024

after fixing this error (commenting out) there is no any visible bluetooth device nearby.

myhomeiot added a commit to myhomeiot/appdaemon-configs that referenced this issue May 4, 2024
@myhomeiot
Copy link
Contributor

@luzik If you don't see Bluetooth device nearby try to use LightBlue or similar application to connect.

@luzik
Copy link

luzik commented May 5, 2024

Thx, yeah it worked. Do you have idea why iPhone do not show it before using LightBlue ? Is there any way to find it without extra app ?

@luzik
Copy link

luzik commented May 5, 2024

Found something like that:
However, it seems that iOS does list BLE devices when they advertise some of the BT SIG adopted services such as the Heart Rate service. By advertising this service you can get your device visible in the Bluetooth Settings page..

Maybe we can change advertising service to be heart rate ?

@luzik
Copy link

luzik commented May 5, 2024

I've changed

auto service_uuid = esphome::esp32_ble::ESPBTUUID::from_uint16(0x1812);
to 0x180D but still not shown. So dead end. I've read that the only way to make it on list is to start "dual mode of BLE and Bluetooth classic"

@luzik
Copy link

luzik commented May 5, 2024

One more thing .. LightBlue shows no services associated with my ble device. Why is that ?

@dgrnbrg
Copy link
Owner

dgrnbrg commented May 5, 2024

The really fun trick of why this works to get the IRK is that I only implemented enough that the BLE handshake happens. Then, there's nothing lol. So we do enough to get the private key, but there isn't anything implemented.

I haven't investigated to understand what/whether there are consequences for this.

dgrnbrg added a commit that referenced this issue May 5, 2024
@myhomeiot
Copy link
Contributor

@luzik Here the dirty hack to advertise service (actually it's advertise 2 same services).
Now it's will works but sure the component should be reviewed/rewritten in order to be included into ESPHome as base service. Maybe it's will requires some Enroll service and only when this service will be visible, maybe it's should send event which will be catches by Private BLE Integration, etc.

@luzik
Copy link

luzik commented May 8, 2024

Worked like a charm! With your hack and 0x180D

@luzik
Copy link

luzik commented May 9, 2024

While my esp is now visible in iPhone and its recover my IRK, is there a way to rename esp devices visible on bluetooth list ? It is defaults to

esphome:
    name:

And this is ok for most of the cases, but If I want to change it, I have to change all the staff (DHCP, HA)

@myhomeiot
Copy link
Contributor

@luzik Today it's not possible to change BLE device name in configuration. You can try another dirty hack, but name can be cached at client devices, so try to wait, enable/disable BT, etc:

void IrkEnrollmentComponent::loop() {
  const char *device_name = "IRK Enroll";
  static bool is_device_name_set = false;
  if (!is_device_name_set) {
    auto dis_ = esp32_ble_server::global_ble_server->get_service(esphome::esp32_ble::ESPBTUUID::from_uint16(0x180A));
    if (dis_ != nullptr) {
      esp32_ble_server::BLECharacteristic *name = dis_->get_characteristic(0x2A00);
      if (name == nullptr) {
        name = dis_->create_characteristic(0x2A00, esp32_ble_server::BLECharacteristic::PROPERTY_READ);
        name->set_value(device_name);
        esp_ble_gap_set_device_name(device_name);
      }
    }
  }
...

@luzik
Copy link

luzik commented May 9, 2024

Thank You again! It is working.
This default name is in one hand ok, because if you have multiple ESP devices in HA, you will know here to lookup for IRK key, but on the other hand, you will install this service only on one ESP device, so configuration even inside of:

irk_enrollment:
  latest_irk:
    name: Latest IRK
    ble_name: IRK Enroll (optional)

would be awesome

@agittins
Copy link
Author

Finally had a chance to poke this again, excited to see the work done!

I wasn't able to directly include your repo as an external component, as esphome has a hard requirement that the components are in a directory named components, not custom_components. I forked it and added a symlink, which seemed to make it happy:

external_components:
  - source: github://agittins/esphome-irk-enrollment
irk_enrollment:
  latest_irk:
    name: Latest IRK

It builds and installs OK, and the esphome shows up in my phone's "Pair new device" listing. But pairing doesn't seem to work for me. Sometimes when I tap the device name, the phone goes to "pairing" for a few milliseconds, then back to the normal state, where I can tap it again. No logs on the esphome when that happens.

Other times when I tap it, the phone (pixel 6pro) sits at the "Pairing..." state, never prompting for the pairing confirmation (the dialog that has the toggle for allowing call/contact access). After 30 seconds it times out and goes back the normal list of devices. Log entries I get at that point include:

[18:59:59][D][esp32_ble_server:155]: BLE Client connected
[19:00:29][D][esp-idf:000][BTU_TASK]: W (1190939) BT_BTM: Security Manager: BTM_SetEncryption not connected
[19:00:32][D][esp-idf:000][BTU_TASK]: W (1193940) BT_HCI: hci cmd send: disconnect: hdl 0x0, rsn:0x13
[19:00:32][D][esp-idf:000][BTU_TASK]: W (1193943) BT_BTM: btm_sec_clr_temp_auth_service() - no dev CB
[19:00:33][D][esp-idf:000][BTU_TASK]: W (1193946) BT_APPL: gattc_conn_cb: if=4 st=0 id=4 rsn=0x16
[19:00:33][D][esp-idf:000][BTU_TASK]: W (1193950) BT_APPL: gattc_conn_cb: if=5 st=0 id=5 rsn=0x16
[19:00:33][D][esp-idf:000][BTU_TASK]: W (1193953) BT_APPL: gattc_conn_cb: if=6 st=0 id=6 rsn=0x16
[19:00:33][D][esp-idf:000][BTU_TASK]: E (1194005) BT_BTM: Device not found
[19:00:33][D][esp-idf:000][BTU_TASK]: W (1194009) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x13
[19:00:33][D][esp-idf:000][BTU_TASK]: W (1194011) BT_APPL: bta_gattc_mark_bg_conn unable to find the bg connection mask for: 6f:21:06:35:c1:94
[19:00:40][D][esp32_ble_tracker:270]: Starting scan...

I suspect that the above could be that the device is missing part of the request, perhaps because it's busy doing other stuff (like sending logs over wifi!). I tried after turning off esp32_ble_tracker scanning (by setting continuous: false), but it didn't seem to make any noticable difference with how often the timeout behaviour happened.

Sometimes, however, when tapping the device to pair, my phone immediately goes to the pairing confirmation dialog, and when I confirm that I get log output like this:

[19:18:13][D][esp32_ble_server:155]: BLE Client connected
[19:18:15][D][esp-idf:000][BTU_TASK]: W (480379) BT_SMP: FOR LE SC LTK IS USED INSTEAD OF STK
[19:18:16][D][esp-idf:000][BTU_TASK]: E (481016) BT_SMP: smp_calculate_link_key_from_long_term_key failed to update link_key. Sec Mode = 2, sm4 = 0x00
[19:18:16][D][esp-idf:000][BTU_TASK]: E (481020) BT_SMP: smp_derive_link_key_from_long_term_key failed
[19:18:16][D][esp-idf:000][BTU_TASK]: E (481028) BT_BTM: btm_proc_smp_cback received for unknown device
[19:18:19][D][esp-idf:000][BTU_TASK]: W (484029) BT_HCI: hci cmd send: disconnect: hdl 0x0, rsn:0x13
[19:18:19][D][esp-idf:000][BTU_TASK]: W (484036) BT_BTM: btm_sec_clr_temp_auth_service() - no dev CB
[19:18:19][D][esp-idf:000][BTU_TASK]: W (484039) BT_APPL: gattc_conn_cb: if=4 st=0 id=4 rsn=0x16
[19:18:19][D][esp-idf:000][BTU_TASK]: W (484041) BT_APPL: gattc_conn_cb: if=5 st=0 id=5 rsn=0x16
[19:18:19][D][esp-idf:000][BTU_TASK]: W (484042) BT_APPL: gattc_conn_cb: if=6 st=0 id=6 rsn=0x16
[19:18:19][D][esp32_ble_server:164]: BLE Client disconnected
[19:18:19][D][esp-idf:000][BTU_TASK]: E (484115) BT_BTM: Device not found
[19:18:19][D][esp-idf:000][BTU_TASK]: W (484118) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x16
[19:18:19][D][esp-idf:000][BTU_TASK]: W (484121) BT_APPL: bta_gattc_mark_bg_conn unable to find the bg connection mask for: 55:8b:ea:6a:5f:8c

at some point this happened, but I suspect it may have been immediately after a mac rotation or something, not sure:

[19:41:41][D][esp32_ble_server:155]: BLE Client connected
[19:41:44][D][esp-idf:000][BTU_TASK]: W (430282) BT_SMP: dhkey checks do no match
[19:41:47][D][esp-idf:000][BTU_TASK]: W (433295) BT_HCI: hci cmd send: disconnect: hdl 0x0, rsn:0x13
[19:41:47][D][esp-idf:000][BTU_TASK]: W (433302) BT_BTM: btm_sec_clr_temp_auth_service() - no dev CB
[19:41:47][D][esp-idf:000][BTU_TASK]: W (433307) BT_APPL: gattc_conn_cb: if=4 st=0 id=4 rsn=0x16
[19:41:47][D][esp-idf:000][BTU_TASK]: W (433309) BT_APPL: gattc_conn_cb: if=5 st=0 id=5 rsn=0x16
[19:41:47][D][esp-idf:000][BTU_TASK]: W (433310) BT_APPL: gattc_conn_cb: if=6 st=0 id=6 rsn=0x16
[19:41:47][D][esp-idf:000][BTU_TASK]: E (433384) BT_BTM: Device not found
[19:41:47][D][esp-idf:000][BTU_TASK]: W (433388) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x16
[19:41:47][D][esp-idf:000][BTU_TASK]: W (433391) BT_APPL: bta_gattc_mark_bg_conn unable to find the bg connection mask for: 7d:cd:4a:66:3f:f0

I found what looked like might be a similar issue and tried adding

    case ESP_GAP_BLE_SEC_REQ_EVT:
        esp_ble_gap_security_rsp(param->ble_security.ble_req.bd_addr, true);
        break;        

to the event handler but it fails to build with (including full build so you can see IDF version etc):

INFO ESPHome 2024.6.6
INFO Reading configuration ./ttgotest.yaml...
INFO Detected timezone 'Australia/Sydney'
WARNING GPIO0 is a strapping PIN and should only be used for I/O with care.
Attaching external pullup/down resistors to strapping pins can cause unexpected failures.
See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins
INFO Generating C++ source...
INFO Compiling app...
Processing ttgotest (board: esp32dev; framework: espidf; platform: platformio/[email protected])
--------------------------------------------------------------------------------
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
 - framework-espidf @ 3.40407.240606 (4.4.7) 
 - tool-cmake @ 3.16.4 
 - tool-ninja @ 1.7.1 
 - toolchain-esp32ulp @ 2.35.0-20220830 
 - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
Reading CMake configuration...
No dependencies
Compiling .pioenvs/ttgotest/src/esphome/components/irk_enrollment/irk_enrollment.o
Compiling .pioenvs/ttgotest/src/esphome/components/socket/socket.o
Compiling .pioenvs/ttgotest/src/esphome/components/spi/spi.o
Compiling .pioenvs/ttgotest/src/esphome/components/spi/spi_arduino.o
Compiling .pioenvs/ttgotest/src/esphome/components/spi/spi_esp_idf.o
Compiling .pioenvs/ttgotest/src/esphome/components/status/status_binary_sensor.o
Compiling .pioenvs/ttgotest/src/esphome/components/switch/automation.o
Compiling .pioenvs/ttgotest/src/esphome/components/switch/switch.o
Compiling .pioenvs/ttgotest/src/esphome/components/text_sensor/filter.o
src/esphome/components/irk_enrollment/irk_enrollment.cpp: In member function 'virtual void esphome::irk_enrollment::IrkEnrollmentComponent::gatts_event_handler(esp_gatts_cb_event_t, esp_gatt_if_t, esp_ble_gatts_cb_param_t*)':
src/esphome/components/irk_enrollment/irk_enrollment.cpp:102:41: error: 'union esp_ble_gatts_cb_param_t' has no member named 'ble_security'
         esp_ble_gap_security_rsp(param->ble_security.ble_req.bd_addr, true);
                                         ^~~~~~~~~~~~
Compiling .pioenvs/ttgotest/src/esphome/components/text_sensor/text_sensor.o
*** [.pioenvs/ttgotest/src/esphome/components/irk_enrollment/irk_enrollment.o] Error 1
========================== [FAILED] Took 7.56 seconds ==========================

At this point I'm messing with the component locally, so the config I have is:

external_components:
  #- source: github://agittins/esphome-irk-enrollment
  #- source: github://dgrnbrg/appdaemon-configs
  - source: esphome-irk-enrollment/components
irk_enrollment:
  latest_irk:
    name: Latest IRK

Here's the full config I'm using, in case I'm doing something daft!

substitutions:
  devicename: ttgotest
  description: TTGO-T-Display board as test platform

esphome:
  name: $devicename
  friendly_name: $devicename
  comment: $description

esp32:
  board: esp32dev
  framework:
    type: esp-idf

logger:
  baud_rate: 0 # disable serial uart logging

ota:
  platform: esphome
  password: !secret ota_password

api:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

switch:
  - platform: restart
    name: Reboot ${devicename}

text_sensor:
external_components:
  #- source: github://agittins/esphome-irk-enrollment
  #- source: github://dgrnbrg/appdaemon-configs
  - source: esphome-irk-enrollment/components
irk_enrollment:
  latest_irk:
    name: Latest IRK

esp32_ble_tracker:
  scan_parameters: 
    active: True
    interval: 320ms # default 320ms. Time spent per adv channel
    window:   300ms # default 30ms. Time spent listening during interval.
    continuous: False

bluetooth_proxy:
  active: True

time:
  - platform: homeassistant
    id: esptime

sensor:
  - platform: wifi_signal
    name: $devicename WiFi Signal
    update_interval: 60s

spi:
  clk_pin: GPIO18
  mosi_pin: GPIO19

color:
  - id: my_red
    red: 100%
    green: 0%
    blue: 0%
  - id: my_yellow
    red: 100%
    green: 100%
    blue: 0%
  - id: my_green
    red: 0%
    green: 100%
    blue: 0%
  - id: my_blue
    red: 0%
    green: 0%
    blue: 100%
  - id: my_gray
    red: 50%
    green: 50%
    blue: 50%

binary_sensor:
  - platform: status
    name: "Node Status"
    id: system_status
  - platform: gpio
    pin:
      number: GPIO0
      inverted: true
      mode:
        input: true
        pullup: true
    name: "T-Display Button Input 0"
    id: tdisplay_button_input_0
  - platform: gpio
    pin:
      number: GPIO35
      inverted: true
    name: "T-Display Button Input 1"
    id: tdisplay_button_input_1
  • Am I doing the right things when trying to build this? (copying the repo, linking the dir)
  • Is there something bone-headed in my config?
  • Do I need to use the arduino framework instead of esp-idf? (I think idf is the default now anyway?)
    • just tested using arduino framework, builds and runs, seems to be the same but without logging any errors, and haven't managed to get the pairing confirmation dialog on the phone after multiple attempts. just get logs saying:
      [21:29:51][D][esp32_ble_server:155]: BLE Client connected
      [21:30:23][D][sensor:094]: 'ttgotest WiFi Signal': Sending state -44.00000 dBm with 0 decimals of accuracy
      [21:30:24][D][esp32_ble_server:164]: BLE Client disconnected
      [21:30:26][D][esp32_ble_server:155]: BLE Client connected
      [21:30:59][D][esp32_ble_server:164]: BLE Client disconnected
      [21:31:20][D][esp32_ble_server:155]: BLE Client connected
      

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants