[BUG] HTTP down

[eskey0]

Faikin hardware
Faikin-S3-MINI-N4-R2: 91c1bc5 2024-03-31T10:59:15 S21 from Amazon

Daikin hardware
FTXP35N5V1B via s403

Describe the bug
The website goes down, I can control the unit via MQTT and ping it, but no HTTP or whatsoever

To Reproduce
No idea, happened out of the blue, I waited to see if it comes back but no dice.

Expected behavior
Have the web service working, I searched for a reboot via MQTT to see if that fix it, but found none.

Additional context
I have 3 of them, all of them configured and setted up the same day, only one of them failed

[revk]

Hmm, odd, we had this ages ago on older code with an app using the legacy URLs, but fixed long ago.

Try just power cycling or sending restart command over MQTT and see if it comes back.

Try web via IP not URL/domain in case an mDNS issue.

[eskey0]

Sorry I didn't specify that yes I do use direct ip address to connect to the device.
After the restart command the website is up again, I don't know if you want to dig more on this, or let it be for now.

[revk]

Ok not sure, as I say, only seen with some very specific (and now fixed) legacy IP polling. See if its happens again.

[eskey0]

Sure, I'll keep an eye on this, and keep you updated, thanks sir you awesome!

[eskey0]

Hello there again, just a heads up, I got my second device to also "http fail", and I, again, fix it by mqtt restart, and now my 3rd device is in that state too.

EDIT: Just wanted to share the status, if no one else is experience this, maybe it's something in my setup

[revk]

Are you using the legacy URLs / polling them?

[eskey0]

I just navigate to http://ipaddress in the browser, it usually just works.

[revk]

OK but no tools, HA plug-ins, or something, that may be accessing the legacy URLs for data?

[eskey0]

No that Iam aware of, just HA through MQTT, nothing going for the HTTP besides my browser that I rarely use.

[revk]

OK, as I know some HA plug-ins use the old URLs, but if using MQTT, that should be fine. Which leaves my rather puzzled at the issue, to be honest.

[eskey0]

It also looks timed, one failed, reboot, about 3/5 days passed by, and then the other one, and repeat. Now is the 3rd one (of 3) I can just reboot it via MQTT too and see if they start from the first one that failed.

To give you more of hindsight, I do have a more-than-average network, the Faikins also are in a restricted network, with some cameras in the same segment, with only access to HA trough MQTT, the web access from my computer, and to your update server.

I do have some plug-ins in HA, but that were for the "official" modules, and they were assigned different IP addresses, and I dissconect them from the units, so I don't think that could be an issue.

[eskey0]

I have more information to share, it happened again, this time to 2 of the 3 devices I have. It happened just after I changed the wifi band on my AP, does that ring any bell? Again after sending a MQTT reboot the website goes online.

I must add I live in an appartment that is very noisy wifi wise.

[antwin]

This has just happened to me. The device is online - responds to pings, nmap can see it but not analyse it, it works on mqtt, but the webserver times out. Addressed by ip address. Webserver is up again after an mqtt restart. Uptime was a few days.
Just before the web server stopped, I was looking at the page. It loaded the first time ok. Then just gave the blue screen with no buttons. On a reload it loaded it all, then timed out.
Faikin-S3-MINI-N4-R2: b16bfc4 2024-08-12T14:02:04 S21
Would any more info help - wireshark capture, status output ... ?

[revk]

Just to check, are you using the legacy URLs? We think, somehow, there is a memory leak, possibly in the ESP IDF.

[antwin]

I'm not sure what you mean by legacy URLs. I'm using the IP address (192.168.0.150) directly.

[revk]

I.e. a monitoring app that talks http to Faikin to get/set data. The way the old Daikin wifi modules used to work.

[antwin]

I'm using Firefox to read from http://192.168.0.150 (the Faikin) on one computer. The page appears to be refreshed at intervals. I have not disconnected the original Daikin wifi module, but that has never been used, and the Daikin app is not available here.

[revk]

OK sounds like you are not using the legacy HTTP API then. The web page on the Faikin is not "refreshed" it uses a web socket. It should have no problem working indefinitely. I'm puzzled if you think it is being refreshed.

When we have seen issues with web server stopping it has always been down to someone using some app (not the Daikin app, usually some home assistant plug in that is not using MQTT). That polls the HTTP legacy APIs constantly, and we think there is some memory leak issue from that, but not 100% sure.

If you are not doing that, it is the first case of a problem like this.

Can you check the settings / basic page occasionally and see if the memory figures on that page are going down over time?

[antwin]

First off, thanks for the prompt replies - I'm very impressed!
My terminology was off. The page is updated, which is why I assumed it was refreshed. I must get the hang of websockets some day.
I'm not using HA. I intend to be using MQTT sometime.
I'll check the memory figures on the settings page, but it's a cold wet night here (NZ) and I'm off to bed, so there will be a pause of a day or two.

[revk]

Have a good night. The fact this is not using legacy HTTP APIs is interesting, and so may give us clues.

[antwin]

Here are some preliminary results from status/faikin - are these what you need to see?:
{"ts":"2024-08-20T05:25:26Z","id":"DC5475EF52FC","up":true,"uptime":3690,"mqtt-up":3686,"mem":119504,"spi":2090296}
{"ts":"2024-08-20T08:28:48Z","id":"DC5475EF52FC","up":true,"uptime":14692,"mqtt-up":14688,"mem":119324,"spi":2090196}
{"ts":"2024-08-20T22:40:31Z","id":"DC5475EF52FC","up":true,"uptime":65794,"mqtt-up":65790,"mem":119120,"spi":2090196}

[revk]

Ah prefect yes mem and SPI, over time.

[antwin]

No http hangs for several days!
More results:
{"ts":"2024-08-21T09:38:24Z","id":"DC5475EF52FC","up":true,"uptime":105267,"mqtt-up":21698,"mem":119324,"spi":2090196}
{"ts":"2024-08-21T23:28:38Z","id":"DC5475EF52FC","up":true,"uptime":155080,"mqtt-up":71511,"mem":118760,"spi":2090196}
{"ts":"2024-08-23T23:37:50Z","id":"DC5475EF52FC","up":true,"uptime":328430,"mqtt-up":244861,"mem":118676,"spi":2090040}
{"ts":"2024-08-25T05:06:19Z","id":"DC5475EF52FC","up":true,"uptime":434538,"mqtt-up":350969} mem 113600+2090108 (for some reason, it's not now reporting "mem" in status.)

[antwin]

MQTT is working fine. BUT although HTTP is working on one device I cannot connect on a second device. Current status:
{"ts":"2024-08-27T22:42:04Z","id":"DC5475EF52FC","up":true,"uptime":670681,"mqtt-up":587112,"mem":109792,"spi":2089848}

[revk]

OK, that means it is not a memory leak. I'll have to look at number of TCP sockets or something.

Does it eventually recover, or does it need a restart?

[antwin]

The working one worked for some hours. But it has also just stopped. It stopped with just the blue background page and 'settings....' at the bottom left, so no updating. So now no http connection on either, but pings and mqtt work fine.

[revk]

This sounds a lot like a TCP related issue. I'll have to have a play with the options.

[Ulf349]

OK. I will report with IP adress, if any http isn´t reachable. Currently all works, give me some days. In the meantime should I upgrade all to BETA?

[revk]

Beta is recommended for any debug as avoids me chasing already fixed issues.

[Obergangster123]

Maybe I can help out with some logs or whatever?
As most of the time mine is currently not reachable at all - also not via IP.

[revk]

None of that HTML is from Faikin!

[Obergangster123]

None of that HTML is from Faikin!

No it's what a chromium based browser displays if there is no response from the faikin module at all. Anyway. If there is something I can help out with like logs or whatever I'm happy to assist. My Faikin goes into this state every other hour.

[revk]

I am not sure what to suggest, sorry

[PianSom]

I think it must have died :(

I ordered a replacement from Amazon, and will hopefully have a chance to replace the old one over the Xmas break. Let me know if you'd like the dead one returned somewhere for a post-mortem

[jinxlal]

I’m facing the same issue described here. At first the faikin homepage takes afew minutes to load, after a couple of hours it stops loading completely. However MQTT seems to be fine as I am able to control the AC thru Home Assistant / Homekit even when the webserver is down. Unit was purchased last week from Amazon UK.

Edit: issue experienced on latest 2024-11 stable as well as 2024-12 beta

[revk]

I am wondering if some issue with the http server code in EDP IDF maybe. This does seem rather odd.

Are you using the legacy Daikin URLs as well?

[revk]

I don't see any settings or options that I can sensibly change.for HTTPD.

There are, however, quite a few TCP settings which can be tweaked. I can reduce some timeouts perhaps. I'll look at doing this in next beta (soon).

[revk]

OK making a beta now, let me know.

[Obergangster123]

I installed the beta. The faikin disappeared from my network and not yet showed up again. Look like its broken now.

[revk]

Err, odd, I tested this on my network and it works. Oh, and now mind has done then same... Hmmm

[revk]

Let me investigate.

[revk]

OK does it appear if you power cycle it. I'm about to try the same.

[revk]

My initial testing worked, the beta did not, which is a problem.

[revk]

OK, the beta is fixed - do you have the means to solder USB leads to the board, to reflash it?

[revk]

This looks like a bug in ESP IDF that has caught me a few times now.

[Obergangster123]

I can solder. I'll check the documentation and will try to fix it in the next days.

[revk]

OK please let me know if any issues, the bottom of https://github.com/revk/ESP32-RevK shows how you can reflash the files using Chrome without having to faff with installing esptool.

[Obergangster123]

Thanks for the hint!

[Obergangster123]

I could not connect to the esp32 yet. Where do I need to connect what lead?
I have a USB to TTL adapter. Alternatively I have a cut USB cable which i can solder somewhere.

Could you please advise where to solder what?
Thanks.

[revk]

It needs direct USB lead connection normally

[Obergangster123]

Means I can solder d- d+ gnd and 5v to the s21 port directly?

[revk]

Err not S21

[Obergangster123]

I must confess i have no idea where to find this "Err" on the board. My board is in 2024-08-06 design. There is the S21 interface and the TC2030 connector which I don't have a cable for. Unfortunately I have not been able to find anything in the circuit board plans in KiCad neither.

[Ulf349]

I am using different modules, but I can help you.
It's only necessary to connect the pin ground means minus to the black wire on the USB cable. The pin near ground is for 5V. That is the red wire.
Then you can use a USB charger. I think the original module have a LED. That should blink or something.
Then you know the power is ok.
The green and white cable from the USB cable is D+ and D-. What is what, use Google.
The special USB connector on the Faikin have this D+and D-.
Solder the cables to the correct connecting point and than your computer show there is a new device connected if you are using Windows 10, I used.
I hope I could help you. Sorry for my German English knowledge.

Datasheet from the esp module.
Find where are both gpio pins.
Use your multimeter and search which pin is where.
Solder the cables.

[revk]

Sorry, err is an expression not an interface, the TC2030 is NOT the S21 interface. It is USB.

[Obergangster123]

Ahh, I thought Err is some kind of bugfix terminal or so :)
Thanks for all your help.(Also Ulf) I appreciate it.
I connected it but it does not show up in Windows. So I'll buy a new one now.

[jinxlal]

Updated my faikin to the latest beta firmware about 36-48 hours ago and the webserver is still responsive. So @revk , whatever you changed seems to have done the trick!

Will update this post in a few days with a confirmation that things are stable or if the issue reappears.

Currently running Faikin-S3-MINI-N4-R2: 5c127c9 2024-12-26T16:33:17 S21