idf.py build problem

[playduck]

I'm using an esp32 and am trying to use the esp32/button example.
I'm using the currently latest stable esp-idf and toolchain on MacOS 10.15.6.
I've cloned the latest version of this repository and initialised all submodules.
All I have modified is adding a valid wifi.h at the repo's root.

I've configured the project using both idf.py menuconfig and make menuconfig by setting a valid serial port as well as allowing dangerous writes in the SPI driver.
The project builds fine with both idf.py build and make all.
The project is also flashed without error using idf.py -p "/dev/tty.SLAB_USBtoUART" flash monitor and make flash monitor.

The problem arises at runtime.
The version of the project using make has no problems at all.

The one build with idf.py however is outputting esp-homekit's debug output, although the option was left on false in the menuconfig. While the accessory shows up in the homekit app, the device crashes upon entering the passcode.
Some notable output of the version using 'idf.py' is:
at startup:

>>> HomeKit: Starting server
!!! HomeKit: Failed to read HomeKit storage magic
>>> HomeKit: Formatting HomeKit storage at 0x200000
!!! HomeKit: Failed to erase HomeKit storage
>>> HomeKit: Resetting HomeKit storage
!!! HomeKit: Failed to reset HomeKit storage
>>> HomeKit: Generated new accessory ID: 65:9D:77:9D:B7:BB
!!! HomeKit: Failed to write accessory ID to HomeKit storage
>>> HomeKit: Generated new accessory key
!!! HomeKit: Failed to write accessory key to HomeKit storage
>>> HomeKit: Configuring mDNS
>>> homekit_run_server: Staring HTTP server

And later, after finishing pairing step 3/3:

!!! HomeKit: Failed to compact HomeKit storage: sector data read error
!!! HomeKit: Failed to write pairing info to HomeKit storage: max number of pairings
!!! HomeKit: [Client 55] Failed to store pairing (code -2)
>>> send_tlv_response: [Client 55] Sending TLV response
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 6 value (1 bytes): \x06
>>> tlv_debug: Type 7 value (1 bytes): \x01
>>> client_send: [Client 55] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0AContent-Length: 6\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A\x06\x01\x06\x07\x01\x01
>>> homekit_client_process: [Client 55] Finished processing
>>> HomeKit: [Client 55] Closing client connection

I could provide more output if needed.
Am I missing something with my usage of the idf.pyor is there some other issue (maybe not even with esp-homekit) here?

[mriksman]

Seems like an issue writing to flash Failed to erase HomeKit storage
Maybe related to what he has on his front page https://github.com/maximkulkin/esp-homekit

In ESP-IDF >= 4.0 there is a SPI flash write protection that checks if area written to is inside writable parition. Haven't figured out yet how esp-homekit can modify parition table automatically, so for the time being you need to disable that check in menuconfig: go to Component config -> SPI Flash driver -> Write to dangerous flash regions and set it to "Allowed".

See if that is the issue.

Although maximkulkin does not appear to be a fan of CMake, it is what is going to be Espressif's only supported system moving forward. You can see some of my notes if it helps. https://github.com/mriksman/esp-idf-homekit/wiki. maximkulkin has included my Pull Request to the CMake files he has.

[playduck]

Thanks for the tip, however I've already set the SPI Flash driver without any change in results.
It's what I meant with:

... as well as allowing dangerous writes in the SPI driver.

I've also tried to run idf.py -p "/dev/tty.SLAB_USBtoUART" erase_flash before reflashing without luck.
This also wouldn't fix the part of the problem where the esp-homekit's debug output is printed despite the menuconfig telling it not to.

I'll be sure to take a look at your wiki. Certainly looks like a good resource. Thanks!

[mriksman]

Maybe it's because idf.py uses a Partition Table?

[playduck]

I'm not sure what a Partition Table entails exactly.
So just to be sure I also tried erasing the flash with make.
I ran make menuconfig to set the serial port (since idf.py overwrites it) and susequently ran
make erase_flash && idf.py -p "/dev/tty.SLAB_USBtoUART" all flash monitor
with no change in result.
Also tried it with two diffrent esp's.

I feel like there's gotta be something I'm missing here either with CMake or idf.py.

[mriksman]

My wiki has details on setting up VSCode and CMake system. And it explains the Partition Table, and how to make a custom table and create a partition for HomeKit data.

I'm not even sure you should be mixing make and idf.py (although, I am not certain). For example, I call 'idf.py menuconfig', not 'make menuconfig'. Not sure if there is a difference?
Read https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/build-system.html. 62 references to idf.py. There is also a link to the 'legacy' Make system. That page mentions nothing of idf.py

I have never used Make, so @maximkulkin is the only one that could help you here.

[WallXiaoming]

same question. I set Flash Size 4MB, everything is ok. idf.py menuconfig -> Serial flasher config -> Flash size -> 4MB