diff --git a/docs/doc/en/sidebar.yaml b/docs/doc/en/sidebar.yaml index 1d86de8b..eea2d943 100644 --- a/docs/doc/en/sidebar.yaml +++ b/docs/doc/en/sidebar.yaml @@ -120,6 +120,8 @@ items: label: RTSP streaming - file: video/rtmp_streaming.md label: RTMP streaming + - file: video/uvc_streaming.md + label: UVC streaming - label: Network items: diff --git a/docs/doc/en/video/uvc_streaming.md b/docs/doc/en/video/uvc_streaming.md new file mode 100644 index 00000000..ec7922dc --- /dev/null +++ b/docs/doc/en/video/uvc_streaming.md @@ -0,0 +1,104 @@ +--- +title: MaixCAM MaixPy Video Streaming UVC Streaming / As a UVC camera to display custom image +update: + - date: 2024-12-20 + author: taorye + version: 1.0.0 + content: 初版文档 +--- + +## Introduction + +`MaixCAM` as a `UVC camera`, where `UVC` stands for `USB video(device) class`. + +Here, two methods are provided to display custom content: + +- Refresh the target image using the `show` method of `maix.uvc.UvcStreamer` (supports YUYV and MJPEG). +- Refresh the target image by registering a custom image refresh callback function with `maix.uvc.UvcServer` (only supports MJPEG). Much more complex than the method above. + + +## Example + +**First, you need to enable the `UVC` function in the `USB settings` section of the `Settings` app.** + +**Note:** +Once the `UVC` function is enabled, due to Linux's implementation of the `UVC Gadget`, a user program is still required to handle `UVC` device events. +Otherwise, the entire `USB` functionality will pause and wait, affecting other simultaneously enabled `Gadget` features like `Rndis` and `NCM`, which may cause network disconnection. +Therefore, for users who also need other `USB` functionalities, it is recommended to use the `UvcStreamer` method when developing UVC display functionality based on `MaixPy`. +Otherwise, ensure that the `MaixCAM` device has other network access methods, such as `WIFI`, to ensure proper development and debugging. + + +### UvcStreamer + +This method does not affect normal USB functionality. The underlying principle is to split the task into two processes. The official implementation uses a `server` process to handle `UVC` device events and encapsulates an easy-to-use, unified image refresh interface `show(img)` for users. You can treat it as a simple `display` linear logic operation. + +**Reference example source code path:** +`MaixPy/examples/vision/streaming/uvc_stream.py` + +### **Example Source (Usage Instructions):** + +1. **Initialize the UvcStreamer object** + +```python +uvcs = uvc.UvcStreamer() +``` + +- (Optional) Switch to MJPEG streaming mode (YUYV default) + +```python +uvcs.use_mjpg(1) +``` + +2. Refresh the image (automatically handles the format, medium performance loss for MJPEG, and high loss for YUYV) + +```python +uvcs.show(img) +``` + +### UvcServer + +This approach offers high performance with a single-process implementation, but USB functionality will only be available when the process is running. Therefore, when stopping this process, it's important to note that the enabled `Rndis` and `NCM` functionalities will temporarily become inactive, causing a network disconnection. + +**Reference example source code path:** +`MaixPy/examples/vision/streaming/uvc_stream.py` + +**Also packaged as an app source code path:** +`MaixCDK/projects/app_uvc_camera/main/src/main.cpp` + +### **Example Source (Usage Instructions):** + +1. **Initialize the UvcServer object (requires a callback function for image refresh)** + +A helper function `helper_fill_mjpg_image` is provided to assist in placing more general `Image` objects into the `UVC` buffer. + +```python +cam = camera.Camera(640, 360, fps=60) # Manually set resolution + # | 手动设置分辨率 + +def fill_mjpg_img_cb(buf, size): + img = cam.read() + return uvc.helper_fill_mjpg_image(buf, size, img) + +uvcs = uvc.UvcServer(fill_mjpg_img_cb) +``` +The reference implementation will `fill_mjpg_img_cb` only trigger a buffer refresh when it returns `0`. +Therefore, it is recommended to use the helper function in the last line: +`return uvc.helper_fill_mjpg_image(buf, size, img)` + +2. Start the UVC, which launches a background thread, non-blocking operation: + +```python +uvcs.run() +``` + +3. Stop the UVC when it's no longer needed. This will restore the background process from the `UvcStreamer` method implementation to ensure normal USB functionality. + +Currently, there is a **BUG** in the MaixPy framework where it forcibly terminates processes upon exit, preventing the functions after the `while not app.need_exit():` loop from being executed, meaning the `stop()` method may not run as expected. +Therefore, for users who require **normal USB functionality**, it is recommended to switch to the `UvcStreamer` method or use the original C++ API from **MaixCDK**. + +**Reference example:** +`MaixCDK/examples/uvc_demo/main/src/main.cpp` + +```python +uvcs.stop() +``` diff --git a/docs/doc/zh/sidebar.yaml b/docs/doc/zh/sidebar.yaml index f06afd07..874e8c41 100644 --- a/docs/doc/zh/sidebar.yaml +++ b/docs/doc/zh/sidebar.yaml @@ -121,6 +121,8 @@ items: label: RTSP 串流 - file: video/rtmp_streaming.md label: RTMP 串流 + - file: video/uvc_streaming.md + label: UVC 串流 - label: 网络通信 items: diff --git a/docs/doc/zh/video/uvc_streaming.md b/docs/doc/zh/video/uvc_streaming.md new file mode 100644 index 00000000..f3d12ffb --- /dev/null +++ b/docs/doc/zh/video/uvc_streaming.md @@ -0,0 +1,94 @@ +--- +title: MaixCAM MaixPy 视频流 UVC 推流 / 化身 UVC 摄像头显示自定义内容 +update: + - date: 2024-12-20 + author: taorye + version: 1.0.0 + content: 初版文档 +--- + +## 简介 + +`MaixCAM` 化身 `UVC 摄像头`, `UVC` 全称为:USB video(device) class,这里提供两种方法供显示自定义内容: + +- 通过 `maix.uvc.UvcStreamer` 的 `show` 方法来刷新目标图片(支持 YUYV 和 MJPEG), +- 通过 `maix.uvc.UvcServer` 注册自定义刷图回调函数来刷新目标图片(仅支持 MJPEG),区别于上一个方法的顺序逻辑,使用有一定的难度 + +## 参考例程 + +首先需要在 `Settings` APP 的 `USB settings` 栏内启用 `UVC` 功能。 + +注意: `UVC` 功能启用后,因为 Linux 的 `UVC Gadget` 实现,仍需一个用户程序处理 `UVC` 设备的事件, +否则整个 `USB` 功能会暂停等待,影响同时启用的其它 `Gadget` 功能,包括 `Rndis` 和 `NCM`,导致断网。 +故对于其它 `USB` 功能有需求的用户,在基于 `MaixPy` 开发 UVC 显示功能时建议采用 `UvcStreamer` 的实现。 +否则请保证 `MaixCAM` 设备有其它联网途径如 `WIFI` 以确保能正常开发调试。 + +### UvcStreamer + +该方法不影响常态下 USB 功能,原理是分了两个进程。官方默认实现了一个 `server` 进程进行`UVC` 设备的事件处理,并封装了易用统一的刷图接口 `show(img)` 供用户使用,当成一个 `display` 线性逻辑操作即可。 + +参考示例源码路径: `MaixPy/examples/vision/streaming/uvc_stream.py` + +示例分析(使用方法): + +1. 初始化 UvcStreamer 对象 + +```python +uvcs = uvc.UvcStreamer() +``` + +- (可选)切换成 MJPEG 模式,默认 YUYV + +```python +uvcs.use_mjpg(1) +``` + +2. 刷图(自动处理格式,MJPEG 中等性能损耗,YUYV 高损耗) + +```python +uvcs.show(img) +``` + + +### UvcServer + +高性能单进程实现,但仅在运行时 USB 全部功能才可用,故停止该进程时需要注意仍启用的 `Rndis` 和 `NCM` 会暂时失效,断开网络链接。 + +参考示例源码路径:`MaixPy/examples/vision/streaming/uvc_stream.py` + +另有封装成 APP 的源码路径:`MaixCDK/projects/app_uvc_camera/main/src/main.cpp` + +示例分析(使用方法): + +1. 初始化 UvcServer 对象,需提供刷图回调函数实现 + +提供了 helper 函数 `helper_fill_mjpg_image` 帮助将更通用的 `Image` 对象刷入 `UVC` 的缓冲区。 + +```python +cam = camera.Camera(640, 360, fps=60) # Manually set resolution + # | 手动设置分辨率 + +def fill_mjpg_img_cb(buf, size): + img = cam.read() + return uvc.helper_fill_mjpg_image(buf, size, img) + +uvcs = uvc.UvcServer(fill_mjpg_img_cb) +``` +`fill_mjpg_img_cb` 参考实现仅当返回 `0` 时,才会正常触发缓冲区刷新。 +故推荐在最后一行使用 helper 函数即可: +`return uvc.helper_fill_mjpg_image(buf, size, img)` + +2. 启动 uvc,后台启动线程,非阻塞 + +```python +uvcs.run() +``` + +3. 停止 uvc,不再使用时需要调用,可恢复 `UvcStreamer` 方法实现中的后台进程,保证 `USB` 功能正常 + +目前有 BUG,MaixPy 框架在退出时会强制终止进程,导致并不能执行完 `while not app.need_exit():` 循环后的函数调用,即该 `stop()` 很难得到执行。 +故对 **保证 `USB` 功能正常** 有需求的用户可以换用 `UvcStreamer` 方法或是移步 `MaixCDK` 的原始 C++ API,参考例程:`MaixCDK/examples/uvc_demo/main/src/main.cpp`。 + +```python +uvcs.stop() +```