add self learn classifier support

docs/doc/en/basic/maixpy_upgrade.md

@@ -19,5 +19,16 @@ If you prefer not to update the system (since system changes are usually minimal
 * Set up WiFi in the settings to connect the system to the internet.
 * Click on `Update MaixPy` in the settings app to proceed with the update.
 
+
+You can also execute Python code to call system command to install:
+```python
+import os
+
+os.system("pip install MaixPy -U")
+```
+
 > If you are comfortable using the terminal, you can also update MaixPy by using `pip install MaixPy -U` in the terminal.
 
+
+
+And you can download `wheel` file (`.whl`format) manually, and send to device(transfer method see [MaixVision Usage](./maixvision.md)), then install by `pip install *****.whl` command.

docs/doc/en/vision/image_ops.md

@@ -64,6 +64,8 @@ MaixPy provides the `maix.image.load` method, which can read images from the fil
 from maix import image
 
 img = image.load("/root/image.jpg")
+if img is None:
+    raise Exception(f"load image failed")
 print(img)
 ```
 

docs/doc/en/vision/self_learn_classifier.md

@@ -4,20 +4,24 @@ title: MaixPy Self-Learning Classifier
 
 ## Introduction to MaixPy Self-Learning Classifier
 
-Typically, to recognize new categories, it is necessary to collect a new dataset and train on a computer, which can be cumbersome and complex. This method eliminates the need for computer-based training, allowing for immediate learning of new objects directly on the device, suitable for less complex scenarios.
+Usually, to recognize new categories, we need to collect a dataset on a computer and retrain the model, which is a cumbersome and difficult process. Here, we provide a method that allows for instant learning of new objects directly on the device without the need for computer-side training, suitable for less complex scenarios.
 
-For example, if there are a drink bottle and a mobile phone in front of you, take a photo of each to serve as the basis for two categories. Then, collect several photos from different angles of each item, extract their features and save them. During recognition, the image's features are compared with the saved feature values, and the closest match determines the classification.
+For example, if there is a bottle and a phone in front of you, you can use the device to take a picture of each as the basis for two classifications. Then, you collect a few more pictures of them from different angles, extract their features and save them. During recognition, the feature values of the image are compared with the saved feature values, and the classification that is more similar to the saved features is considered the corresponding classification.
 
 ## Using the Self-Learning Classifier in MaixPy
 
-Steps:
+The default image comes with the [Self-Learning Classification APP](https://maixhub.com/app/30), which you can use directly to get familiar with the process.
+
+![](../../assets/self_learn_classifier.jpg)
 
-* Collect n classification images.
-* Collect n*m images, m images for each category, order does not matter.
-* Start learning.
-* Recognize images and output results.
+Steps:
+* Click the `+ Class` button to collect n classification (class) images. The object needs to be within the white frame on the screen while collecting the images.
+* Click the `+ Sample` button to collect m sample images. Collect some images for each classification. The order does not matter, and the number is flexible. It's best to take pictures from different angles, but not too different.
+* Click the `Learn` button to start learning. The device will automatically classify and learn based on the collected classification and sample images, obtaining the characteristics of the classifications.
+* Align the object with the center of the screen, recognize the image, and output the result. The screen will show the classification it belongs to and the similarity distance to this classification. The closer the similarity distance, the more similar it is.
+* The feature values ​​learned by this APP will be saved to `/root/my_classes.bin`, so the last one will be automatically loaded after exiting the application or restarting it.
 
-Simplified version of the code, for the full version please refer to the complete code in the example.
+Simplified version of the code, for the complete version, please refer to the [examples](https://github.com/sipeed/maixpy/tree/main/examples/vision/ai_vision) for the full code.
 
 ```python
 from maix import nn, image
@@ -34,7 +38,6 @@ sample_4 = image.load("/root/sample_4.jpg")
 sample_5 = image.load("/root/sample_5.jpg")
 sample_6 = image.load("/root/sample_6.jpg")
 
-
 classifier.add_class(img1)
 classifier.add_class(img2)
 classifier.add_class(img3)
@@ -51,3 +54,20 @@ img = image.load("/root/test.jpg")
 max_idx, max_score = classifier.classify(img)
 print(max_idx, max_score)
 ```
+
+## Storing and Loading Learned Feature Values
+
+Use the `save` function to store the learned feature values. This will generate a binary file containing the feature values of the objects. When you need to use it again, simply use the `load` function to load the feature values.
+
+```python
+classifier.save("/root/my_classes.bin")
+classifier.load("/root/my_classes.bin")
+```
+
+If you have named each classification and stored them in the `labels` variable, you can also use:
+
+```python
+classifier.save("/root/my_classes.bin", labels=labels)
+labels = classifier.load("/root/my_classes.bin")
+```
+

docs/doc/zh/basic/maixpy_upgrade.md

@@ -20,6 +20,14 @@ title: 更新 MaixPy
 * 在设置中设置 WiFi， 让系统联网。
 * 点击设置应用中的 `更新 MaixPy` 进行更新。
 
+也可以执行 Python 代码调用系统命令来更新：
+```python
+import os
 
-> 如果你会使用终端， 也可以在终端中使用 `pip install MaixPy -U` 来更新 MaixPy。
+os.system("pip install MaixPy -U")
+```
+
+> 如果你会使用终端， 也可以直接在终端中使用 `pip install MaixPy -U` 来更新 MaixPy。
+
+另外你也可以手动下载`wheel` 文件（`.whl`格式）传输到设备（传输方法见后文[MaixVision 使用](./maixvision.md)）后通过 `pip install ******.whl` 命令来安装。
 

docs/doc/zh/vision/image_ops.md

@@ -64,6 +64,8 @@ MaixPy 提供了`maix.image.load`方法，可以从文件系统读取图像：
 from maix import image
 
 img = image.load("/root/image.jpg")
+if img is None:
+    raise Exception(f"load image failed")
 print(img)
 ```
 

docs/doc/zh/vision/self_learn_classifier.md

@@ -12,14 +12,19 @@ title: MaixPy 自学习分类器
 
 ## MaixPy 中使用自学习分类器
 
+默认镜像自带了 [自学习分类 APP](https://maixhub.com/app/30)，可以直接尝试使用熟悉使用流程。
+
+![](../../assets/self_learn_classifier.jpg)
+
 步骤：
-* 采集 n 张分类图。
-* 采集 n*m 张图，每个分类采集 m 张，顺序无所谓。
-* 启动学习。
-* 识别图像输出结果。
+* 点击`+ Class` 按钮， 采集 n 张分类(class)图，采集图时物体需要在屏幕的白色框中。
+* 点击`+ Sample`按钮，采集 m 张样本图，每个分类都采集一些，顺序无所谓，张数也比较随意，最好是在各个角度拍一点，不要差距过大。
+* 点击`Learn`按钮，启动学习，会自动根据采集的分类图和样本图进行分类学习，得到分类的特征。
+* 屏幕中央对准物体，识别图像输出结果，可以看到屏幕显示了所属的分类，以及和这个分类的相似距离，相似距离越近则越相似。
+* 此 APP 学习后的特征值会存到`/root/my_classes.bin`，所以退出应用或者重启了仍然会自动加载上一次的。
 
+简洁版本代码，完整版本请看[例程](https://github.com/sipeed/maixpy/tree/main/examples/vision/ai_vision)里面的完整代码。
 
-简洁版本代码，完整版本请看例程里面的完整代码。
 ```python
 from maix import nn, image
 
@@ -53,5 +58,20 @@ max_idx, max_score = classifier.classify(img)
 print(maix_idx, max_score)
 ```
 
+## 储存和加载学习到的特征值
+
+使用 `save` 函数进行储存，会得到一个二进制文件，里面存了物体的特征值。
+再使用时用`load`函数进行加载即可。
+
+```python
+classifier.save("/root/my_classes.bin")
+classifier.load("/root/my_classes.bin")
+```
+
+如果你给每一个分类命名了，比如存到了`labels`变量，也可以使用：
+```python
+classifier.save("/root/my_classes.bin", labels = labels)
+labels = classifier.load("/root/my_classes.bin")
+```
 
 

examples/vision/ai_vision/nn_self_learn_classifier.py

@@ -0,0 +1,64 @@
+from maix import nn, image, display, app, time
+
+disp = display.Display()
+classifier = nn.SelfLearnClassifier(model="/root/models/mobilenet_v2_no_top.mud")
+
+class_path = [
+    "/root/1.jpg",
+    "/root/2.jpg",
+    "/root/3.jpg"
+]
+
+samples_path = [
+    "/root/sample_1.jpg",
+    "/root/sample_2.jpg"
+]
+
+
+class_images = []
+for path in class_path:
+    # load image from file
+    img = image.load(path)
+    if img is None:
+        raise Exception(f"load image {path} failed")
+    class_images.append(img)
+    # add new class
+    classifier.add_class(img)
+
+sample_images = []
+for path in samples_path:
+    # load image from file
+    img = image.load(path)
+    if img is None:
+        raise Exception(f"load image {path} failed")
+    sample_images.append(img)
+    # add new class
+    classifier.add_sample(img)
+
+if len(sample_images) > 0:
+    print("-- start learn")
+    classifier.learn()
+    print("-- learn complete")
+
+classifier.learn()
+
+img = image.load("/root/test.jpg")
+result = classifier.classify(img)
+print(f"distances: {result}")
+print(f"min distance idx: {result[0][0]}, distance: {result[0][1]}")
+
+# show result
+img_show = image.Image(disp.width(), disp.height())
+img = img.resize(disp.width() // 2, disp.height() // 2, image.Fit.FIT_CONTAIN)
+img_show.draw_image(0, 0, img)
+img_res = class_images[result[0][0]].resize(disp.width() // 2, disp.height() // 2, image.Fit.FIT_CONTAIN)
+img_show.draw_image(disp.width() // 2, 0, img_res)
+img_show.draw_string(2, disp.height() // 2 + 80, f"distance: {result[0][1]}", scale=1.5)
+img_show.draw_string(2, disp.height() // 2 + 40, f"test.jpg", scale=1.5)
+img_show.draw_string(disp.width() // 2 + 2, disp.height() // 2 + 40, class_path[result[0][0]], scale=1.5)
+
+disp.show(img_show)
+
+
+while not app.need_exit():
+    time.sleep_ms(100)

examples/vision/ai_vision/nn_self_learn_classifier_cam.py

@@ -0,0 +1,3 @@
+
+# see https://github.com/sipeed/maixpy/tree/main/projects/app_self_learn_classifier
+

examples/vision/image_basic/binary.py

@@ -2,6 +2,8 @@
 
 # 1. load image
 src_img = image.load("test.jpg")
+if src_img is None:
+    raise Exception(f"load image {file_path} failed")
 
 # 2. binarize the image
 thresholds = ((0, 100, 20, 80, 10, 80))

examples/vision/image_basic/image_load.py

@@ -2,6 +2,8 @@
 
 file_path = "/maixapp/share/icon/detector.png"
 img = image.load(file_path, format = image.Format.FMT_RGBA8888 if file_path.endswith(".png") else image.Format.FMT_RGB888)
+if img is None:
+    raise Exception(f"load image {file_path} failed")
 
 disp = display.Display()
 disp.show(img, fit = image.Fit.FIT_CONTAIN)

maix/maix_resize.py

@@ -15,6 +15,8 @@ def main_cli():
 
     from maix import image
     img = image.load(args.input)
+    if img is None:
+        raise Exception(f"load image {args.input} failed")
     fit = {
         'fill': image.Fit.FIT_FILL,
         'contain': image.Fit.FIT_CONTAIN,

maix/v1/image.py

@@ -42,6 +42,8 @@ def __init__(self, path=None, copy_to_fb=False, width=640, height=480, do_nothin
                 self.__img = image.Image(width, height)
             else:
                 self.__img = image.load(path)
+                if self.__img is None:
+                    raise Exception(f"load image {path} failed")
 
     def get_priv_img(self):
         return self.__img

maix/version.py

@@ -3,6 +3,6 @@
 
 version_major = 4
 version_minor = 3
-version_patch = 0
+version_patch = 2
 
 __version__ = "{}.{}.{}".format(version_major, version_minor, version_patch)

projects/app_self_learn_classifier/.gitignore

@@ -0,0 +1,5 @@
+
+build
+dist
+/CMakeLists.txt
+

projects/app_self_learn_classifier/app.yaml

@@ -0,0 +1,10 @@
+id: self_learn_classifier
+name: Self Learn Classifier
+version: 1.0.1
+author: Sipeed Ltd
+icon: app.png
+desc: Learn anything on device, no PC training needed.
+files:
+  - app.png
+  - app.yaml
+  - main.py