esp32cam - updated driver - ready for help & testing.

[btsimonh]

Hi All,

I've spent the last few weeks working on the webcam driver.
It now has all the features I could think of easily - the ones that I thought could be used from Berry scripting.
It's very stable on my (notoriously unreliable) cam.

It's here:
https://github.com/btsimonh/Tasmota/blob/webcam2023/tasmota/tasmota_xdrv_driver/xdrv_99_esp32_webcamberry.ino

But I don't want to PR until we've got some feedback, and maybe additional ideas taken into account.

VERY brief overview:
It runs all frame grabbing in a thread, so no more possible conflicts, and also a bit more performance.
I've added TAS commands for some of the functions that were previpously only available to scripts (not to berry...).
I've added some image conversion as TAS commands.
It SHOULD still work with (non-berry) scripting (apart from maybe the USE_WEBCAM #define).

Exxample: you can get actual pixels in berry by
WcGetFrame1 (reads the next frame to buffer)
WcConvertFrame1 5 0 (decode the JPEG buffer to RGB888 at scale 1:1)
Wcgetpicstore1 (returns the actual buffer address, length, width, height, format).
(you can download the raw pixels at this point from http://wc.jpeg?p=1)

You could then draw on the image!.... and then
WcConvertFrame1 4 0 (encode to JPEG)
access the resulting jpeg at http://wc.jpeg?p=1

Or... you could use the image in Tensorflow....

It also implements three forms of motion detection:
1/ JPEG size change. No performance Impact (i.e. will run at the cam framerate)
2/ accumulated pixel difference (as per original motion, only accessible through scripts).
3/ pixel difference threshold -> count of pixels over threshold.
The motion detection is somehwat faster (300ms vs 1.3s for a large frame) due to a modified JPEG decode to decode direct to monochrome).

You can access the motion buffers (re-encoded to jpeg) via the browser, and enable a diff buffer.
You can access the motion pixels with wcGetMotionPixels (gives addr and len - for berry?)
If informs berry of motion (webcan.motion is called if present).

If it gets an error at start, the error is displayed in the main menu.
You can turn off and on menu video (streaming takes some bandwidth... if you don't want to, you don't need to).

I still get some 0x105 at soft restart occasionally - the cross to bear from having an AIThinker cam module. But it's rare.

I did try to get raw pixels from the camera (more efficient if that's all you want), but failed.

So, what would you do with such features in a Berry script? What's missing?

Any testing, ideas, thoughts welcome.

br,
Simon

[btsimonh]

I've added a masking image to motion detection - with 'automask' (sets mask from next N motion sense runs).
The mask pixels are available from a command, so in theory, you could set them from Berry.
However, most likely we'd like to load them from file. This is something Berry could be good at (determining what to load, and where to put it). But we'd need some more image manipulation functions.
e.g. currently we can get jpeg from the camera, and berry can read and access the raw data.
Berry can convert such images (in place) to raw pixels, and back to jpeg.
But we can't (at the moment) read a jpeg from disk in berry, and set it into one of the 4 'user' buffers in the webcam driver, ready for manipulation. And the berry access to image data is a little 'knife and fork) (passing raw addresses and length as command repsonses...).

enable/set mask

WCsetMotiondetect8 0-nnnnn
WCsetMotiondetect8 0 - free
WCsetMotiondetect8 1 - alloc/create
WCsetMotiondetect8 n - alloc/create and set automask to n, pixel threshold to 10, pixel expansion to 4
WCsetMotiondetect8 n m o - alloc/create and set automask to n, pixel threshold to m, pixel expansion to o

[btsimonh]

Brief comparison of the two drivers:

tasmota32-webcam 13.1.0 - current release

Params used: AIThinker style, wcresolution 5, wcclock 20
Web: ~20fps
Single web session only.
Single RTSP stream only.

Motion only acessible by using scripts (special build).
Image save only accessible using scripts.
Only one form of motion detection available.

tasmota32-webcam PR

PR binary: tasmota32-webcam-PR.zip
Docs
Example Timelapse and Motion
Params used: AIThinker style, wcresolution 5, wcclock 20
Web: ~35fps

Multiple web sessions available. (with 2 running ~20 fps)
Multiple RTSP sessions available.
Video in main menu can be turned on/off, whilst still processing images.
Berry scripting support by default.

Additional features (some new, some based on existing script related code)
Motion available via TAS commands.
Image save to FS (flash or SD) available as TAS commands.
Berry triggers on frame(if enabled) and/or motion.
Image jpg decode and encode available as TAS commands (up to four slots for image memory - based on scripts implementation)
Images can be read from berry - jpeg or raw pixels (e.g. to post to website, manipulate, use with AI, etc.)
Camera can be disabled using wcresolution -1

Advanced additional features:
Motion detection tunable (scale of jpeg pixels configurable, jpeg size triggers, mask image may be used to mask out moving e.g. trees/road)

thing that the new driver enables people to do:

timelapse with mutliple possibilites

• e.g. write to SD and view directly with static webserving enabled and a simple webpage on the SD card.
• e.g. post images and data about them to a node-red flow, or other simple webserver.
• berry could be used with ULP to deepsleep between pictures. i.e. a battery powered timelapse becomes possible.
  timelapse AND motion at the same time. multiple timelapse configurations at the same time.
  RTSP to other video, without the risk of the RTSP session terminating because of a second connection.
  MJPEG to other video, without the risk of the MJPEG session terminating because you launch the web interface.
  It enables the future use of AI image analysis.

[pkkrusty]

Thanks for all the work on this. I'll get a few modules flashed and play with them for a while.

[btsimonh]

It would be useful to understand all the existing use cases for ESP32cam - that could advise on the focus for any documentation changes and example Berry snippets, etc.

[tablatronix]

Are there any berry examples for this?

[sfromis]

With a quick search, I found a couple of places with a bit of code
https://tasmota.github.io/docs/Berry/#cam-module
https://tasmota.github.io/docs/Berry/#img-class
https://tasmota.github.io/docs/TFL/#berry-implementation
https://github.com/fknaopen/Berry_webcam_motion
https://github.com/tasmota/Berry_playground/blob/main/modules/googleapis/examples/gdrivepics.be

[btsimonh]

Are there any berry examples for this?

also

https://github.com/btsimonh/TasmotaBerryPLay/tree/master/timelapse

check the issues page for a recent question ref current dev building of the updated driver & timelapse.

[tablatronix]

Thanks I have been trying to get getFrame and getStored working and there seems to be several issues with the picstore buffer. So gonna just look into straight berry or a driver

[mrneutron42]

Hi,
I'm curious if these fixes stop the InitErr 0x105 error after a soft reboot (restart 1) command?
reference: #18492

09:09:53.637 CMD: wcinit
09:09:53.677 CAM: InitErr 0x105 try 1
09:09:53.714 CAM: InitErr 0x105 try 2
09:09:53.751 CAM: InitErr 0x105 try 3
09:09:53.752 CAM: InitErr 0x105
09:09:53.789 CAM: InitErr 0x105 try 1
09:09:53.826 CAM: InitErr 0x105 try 2
09:09:53.863 CAM: InitErr 0x105 try 3
09:09:53.864 CAM: InitErr 0x105

[sfromis]

IME, it does not make the bad hardware work.

[tablatronix]

@mrneutron42 those look like camera init issues, check your gpio 0 or 16 on that board. My issue is that pic buffer does not allocate on non psram or my heap is too small not sure yet

[mrneutron42]

I've got some the the ESP32-CAM boards with the DM on the metal cover
https://www.amazon.com/gp/product/B096WBMZ43/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&th=1
and one of them has been giving the InitErr 0x105 when I issue the 'wcinit' command.

I have the camera in a remote location and a 'restart 1' command is my only option of restarting it from a distance.
Doing the 'restart 1' soft restart has not been enough to recover the functionality.

This has happened in the past with this particular camera.
I don't have any wires connected to the pin headers, except for 5V and GROUND.
I think my workaround was to physically pull the power plug to the camera and do a hard reboot.
It recovered the camera for a few months, but the problem occurred again.

[tablatronix]

I just ordered some, is this the board that doesnt have a camera reset line? That would explain it, you would have to add some kind of pmic or watchdog to hard reset

[mrneutron42]

I think this is the schematic for the ESP32 camera board:
https://github.com/SeeedDocument/forum_doc/raw/master/reg/ESP32_CAM_V1.6.pdf
I see a FET connected to GPIO32 which switches the power to the CSI 2.8V and CSI 1.2V power rails.
So I think my boards would respond to the power switching GPIO.

[mrneutron42]

In this post #18492 (comment)
btsimonh said he extended the reset period from 10ms to 200ms, which sounded like it helps.
@btsimonh, I'd like to try the revised firmware and see if it helps my cameras.

[mrneutron42]

I just made a trip to my remote camera site and simply did a 'hard reset' (pulled the power for 10 seconds) and the camera was recovered - no more 'InitErr 0x105' errors upon repower!
To recap, doing a remote "soft reset" (issuing the 'restart 1' command) did not work.
Conclusion: this is not a physical connection problem with my ESP32-CAM camera.

I suspect that the longer (200mS) power down in the modified code would help my recover my cameras, when they go stupid.