-
Notifications
You must be signed in to change notification settings - Fork 43
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
14 changed files
with
715 additions
and
90 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
--- | ||
title: MaixPy 视频流 JPEG 推流 / 发送图片到服务器 | ||
update: | ||
- date: 2024-04-03 | ||
author: neucrack | ||
version: 1.0.0 | ||
content: 初版文档 | ||
--- | ||
|
||
## 简介 | ||
|
||
有时需要将图像发送到服务器,或者将摄像头的视频推送到服务器,这里提供一个最简单的方法,即压缩成 `JPEG` 图片,然后一张一张地发送到服务器。 | ||
|
||
注意,这是一种最简单的方法,不算很正规的视频推流方法,也不适合高分辨率高帧率的视频流,因为这只是一张一张发送图片,如果要高效推送视频流,请使用后文的 `RTSP` 或者 `RTMP` 模块。 | ||
|
||
## 使用方法 | ||
|
||
```python | ||
from maix import image | ||
import requests | ||
|
||
# create image | ||
img = image.Image(640, 480, image.Format.FMT_RGB) | ||
# draw something | ||
img.draw_rect(60, 60, 80, 80, image.Color.from_rgb(255, 0, 0)) | ||
|
||
# convert to jpeg | ||
jpeg = img.to_format(image.Format.FMT_JPEG) # image.Format.FMT_PNG | ||
# get jpeg bytes | ||
jpeg_bytes = jpeg.to_bytes() | ||
|
||
# faster way, borrow memory from jpeg object, | ||
# but be carefully, when jpeg object is deleted, jpeg_bytes object MUST NOT be used, or program will crash | ||
# jpeg_bytes = jpeg.to_bytes(copy = False) | ||
|
||
# send image binary bytes to server | ||
url = "http://192.168.0.123:8080/upload" | ||
res = requests.post(url, data=jpeg_bytes) | ||
print(res.status_code) | ||
print(res.text) | ||
``` | ||
|
||
可以看到,先将图片转换成了 `JPEG` 格式,然后将 `JPEG` 图片的二进制数据通过`TCP`发送到服务器。 | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
--- | ||
title: AI 视觉基本知识 | ||
update: | ||
- date: 2024-04-03 | ||
author: neucrack | ||
version: 1.0.0 | ||
content: 初版文档 | ||
--- | ||
|
||
|
||
## 简介 | ||
|
||
如果没有 AI 基础,在学习 AI 前可以先看[什么是人工智能(AI)和机器学习](https://wiki.sipeed.com/ai/zh/basic/what_is_ai.html) 了解一下 AI 的基本概念。 | ||
|
||
然后我们使用的视觉 AI 一般都是基于`深度神经网络学习`这个方法,有兴趣可以看看[深度神经网络(DNN)基础知识](https://wiki.sipeed.com/ai/zh/basic/dnn_basic.html) | ||
|
||
|
||
## MaixPy 中使用视觉 AI | ||
|
||
在 MaixPy 中使用视觉 AI 很简单,默认提供了常用的 AI 模型,不需要自己训练模型就可以直接使用,在[MaixHub 模型库](https://maixhub.com/model/zoo) 中选择`maixcam` 就可以找到。 | ||
|
||
并且在底层已经封装好的 API,只需要简单的调用就可以实现。 | ||
|
||
如果你想训练自己的模型,也可以先从[MaixHub 在线训练](https://maixhub.com/model/training/project) 开始,在线平台只需要点点点就能训练出模型,不需要购买昂贵的机器,不需要搭建复杂的开发环境,也不需要写代码,非常适合入门,也适合懒得翻代码的老手。 | ||
|
||
一般训练得到了模型文件,直接传输到设备上,调用 MaixPy 的 API 就可以使用了,具体的调用方法看后文。 | ||
|
||
|
||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
--- | ||
title: MaixPy 摄像头使用 | ||
update: | ||
- date: 2024-04-03 | ||
author: neucrack | ||
version: 1.0.0 | ||
content: 初版文档 | ||
--- | ||
|
||
|
||
## 简介 | ||
|
||
对于 MaixCAM 默认搭载了 GC4653 摄像头,或者可选的 OS04A10 摄像头或者全局快门摄像头,甚至是 HDMI 转 MIPI 模块,都可以直接用简单的 API 调用。 | ||
|
||
## API 文档 | ||
|
||
本文介绍常用方法,更多 API 使用参考 [maix.camera](/api/maix/camera.html) 模块的文档。 | ||
|
||
## 摄像头切换 | ||
|
||
不同的摄像头使用不同的驱动,需要在系统中选择正确的驱动。 | ||
|
||
TODO:如何切换摄像头,比如 GC4653 和 OS04A10 之间的切换。 | ||
|
||
|
||
## 从摄像头获取图像 | ||
|
||
使用 MaixPy 轻松获取: | ||
```python | ||
from maix import camera | ||
|
||
cam = camera.Camera(640, 480) | ||
|
||
while 1: | ||
img = cam.read() | ||
print(img) | ||
``` | ||
|
||
这里我们从`maix`模块导入`camera`模块,然后创建一个`Camera`对象,指定图像的宽度和高度。然后在一个循环中不断读取图像, 默认出的图为`RGB`格式,如果需要`BGR`格式,其它格式请看 API 文档。 | ||
|
||
## 跳过 开头的帧 | ||
|
||
摄像头初始化的一小段时间,可能图像采集还没稳定出现奇怪的画面,可以通过`skip_frames`函数跳过开头的几帧: | ||
```python | ||
cam = camera.Camera(640, 480) | ||
cam.skip_frames(30) # 跳过开头的30帧 | ||
``` | ||
|
||
## 显示图像 | ||
|
||
MaixPy 提供了`display`模块,可以方便的显示图像: | ||
```python | ||
from maix import camera, display | ||
|
||
cam = camera.Camera(640, 480) | ||
disp = display.Display() | ||
|
||
while 1: | ||
img = cam.read() | ||
disp.show(img) | ||
``` | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
--- | ||
title: MaixPy 使用 AI 模型进行物体分类 | ||
--- | ||
|
||
## 物体分类概念 | ||
|
||
比如眼前有两张图片,一张图里面是苹果,另一张是飞机,物体分类的任务就是把两张图分别依次输入给 AI 模型,模型会依次输出两个结果,一个是苹果,一个是飞机。 | ||
|
||
## MaixPy 中使用物体分类 | ||
|
||
MaixPy 默认提供了 `imagenet` 数据集训练得到的 `1000`分类模型,可以直接使用: | ||
```python | ||
from maix import camera, display, image, nn | ||
|
||
classifier = nn.Classifier(model="/root/models/mobilenetv2.mud") | ||
cam = camera.Camera(classifier.input_width(), classifier.input_height(), classifier.input_format()) | ||
dis = display.Display() | ||
|
||
while 1: | ||
img = cam.read() | ||
res = classifier.classify(img) | ||
max_idx, max_prob = res[0] | ||
msg = f"{max_prob:5.2f}: {classifier.labels[max_idx]}" | ||
img.draw_string(10, 10, msg, image.COLOR_RED) | ||
dis.show(img) | ||
``` | ||
|
||
效果视频: | ||
|
||
<video playsinline controls autoplay loop muted preload src="https://wiki.sipeed.com/maixpy/static/video/classifier.mp4" type="video/mp4"> | ||
Classifier Result video | ||
</video> | ||
|
||
这里使用了摄像头拍摄图像,然后传给 `classifier`进行识别,得出结果后,将结果显示在屏幕上。 | ||
|
||
更多 API 使用参考 [maix.nn](/api/maix/nn.html) 模块的文档。 | ||
|
||
|
||
## 训练自己的分类模型 | ||
|
||
请到[MaixHub](https://maixhub.com) 学习并训练分类模型,创建项目时选择`分类模型`即可。 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
--- | ||
title: MaixPy 找色块 | ||
update: | ||
- date: 2024-04-03 | ||
author: neucrack | ||
version: 1.0.0 | ||
content: 初版文档 | ||
--- | ||
|
||
## 简介 | ||
|
||
在视觉应用中,找色块是一个非常常见的需求,比如机器人找色块,自动化生产线找色块等等。 | ||
即需要识别画面中的特定的颜色区域,获取这个区域的位置和大小等信息。 | ||
|
||
|
||
## 使用设备自带的找色块应用 | ||
|
||
打开设备,选择`找色块`应用,然后在下方选择要识别的颜色,或者自定义颜色,即可以识别到对应的颜色了,同时串口也会输出识别到的坐标和颜色信息。 | ||
|
||
<video src="/static/video/find_blobs.mp4" controls="controls" width="100%" height="auto"></video> | ||
|
||
### 自定义颜色的方法 | ||
|
||
TODO: | ||
|
||
### 串口协议 | ||
|
||
TODO: | ||
|
||
## 使用 MaixPy 找色块 | ||
|
||
`maix.image.Image`中提供了`find_blobs`方法,可以方便的找色块。 | ||
|
||
```python | ||
from maix import image, camera, display | ||
|
||
cam = camera.Camera(320, 240) | ||
disp = display.Display() | ||
|
||
thresholds = [[0, 100, -120, -10, 0, 30]] | ||
|
||
while 1: | ||
img = cam.read() | ||
blobs = img.find_blobs(thresholds) | ||
for blob in blobs: | ||
img.draw_rectangle(blob[0], blob[1], blob[2], blob[3], color=(255, 0, 0)) | ||
disp.show(img) | ||
``` | ||
|
||
这里的 `thresholds` 是一个颜色阈值列表,每个元素是一个颜色阈值,同时找到多个阈值就传入多个,每个颜色阈值的格式为 `[L_MIN, L_MAX, A_MIN, A_MAX, B_MIN, B_MAX]`,这里的 `L`、`A`、`B` 是`LAB`颜色空间的三个通道,`L` 通道是亮度,`A` 通道是红绿通道,`B` 通道是蓝黄通道。 | ||
可以在上面使用`找色块`应用中找到被检测物体对应的颜色阈值。 | ||
|
||
更多参数和用法请参考 API 文档。 | ||
|
||
|
Oops, something went wrong.