Lite.AI 🚀🚀🌟

Introduction.

Lite.AI 🚀🚀🌟 is a simple and user-friendly C++ library of awesome🔥🔥🔥 AI models. It's a collection of personal interests. such as YOLOX, YoloV5, YoloV4, DeepLabV3, ArcFace, etc. Lite.AI based on onnxruntime c++ by default. I do have plans to reimplement it with ncnn and MNN, but not coming soon. It includes object detection, face detection, style transfer, face alignment, face recognition, segmentation, colorization, face attributes analysis, image classification, matting, etc. You can use these awesome models simply through lite::cv::Type::Class syntax, such as lite::cv::detection::YoloV5. Star 🌟👆🏻 this repo if it does any helps to you ~ Have a good travel ~ 🙃🤪🍀

Important Notes !!!

⚠️ (20210802) Added GPU Compatibility for CUDAExecutionProvider. issue#10.
⚠️ (20210801) fixed issue#9 YOLOX inference error for non-square shape. See yolox.cpp.
✅ (20210731) Added RetinaFace-CVPR2020 for face detection, 1.6Mb only! See demo.
🔥 (20210728) Added FaceLandmarks1000 for 1000 facial landmarks detection, 2Mb only! See demo.
✅ (20210722) Update lite.ai.hub.onnx.md ! Lite.AI contains 60+ AI models with 100+ .onnx files now.
🔥 (20210721) Added YOLOX to Lite.AI ! Use it through lite::cv::detection::YoloX syntax ! See demo.

Expand for More Notes.

More Notes !!!

✅ (20210801) Added FaceBoxes for face detection! See demo.
✅ (20210727) Added MobileNetV2SE68、PFLD68 for 68 facial landmarks detection! See demo.
✅ (20210726) Added PFLD98 for 98 facial landmarks detection! See demo.
⚠️ (20210716) Lite.AI was rename from the LiteHub repo! LiteHub will no longer be maintained.

Working Notes. 👇🏻

✅ object detection
✅ image classification
❇️ face detection
✅ face alignment
✅ face recognition
✅ face attributes analysis
⚠️ segmentation
⚠️ style transfer
⚠️ colorization
⚠️ matting

1. Dependencies.

Mac OS.

install OpenCV and onnxruntime libraries using Homebrew or you can download the built dependencies from this repo. See third_party and build-docs¹ for more details.

  brew update
  brew install opencv
  brew install onnxruntime

Expand for More Details of Dependencies.

Linux.

todo⚠️

Windows.

todo⚠️

Inference Engine Plans:

doing:
❇️ onnxruntime
todo:
⚠️ NCNN
⚠️ MNN
⚠️ OpenMP

2. Build Lite.AI.

Build the shared lib of Lite.AI for MacOS from sources. Note that Lite.AI uses onnxruntime as default backend, for the reason that onnxruntime supports the most of onnx's operators.

Linux and Windows.

⚠️ Lite.AI is not directly support Linux and Windows now. For Linux and Windows, you need to build or download(if have official builts) the shared libs of OpenCV and ONNXRuntime firstly and put then into the third_party directory. Please reference the build-docs¹ for third_party.

Windows: You can reference to issue#6
Linux: The Docs and Docker image for Linux will be coming soon ~ issue#2
Happy News !!! : 🚀 You can download the latest ONNXRuntime official built libs of Windows, Linux, MacOS and Arm !!! Both CPU and GPU versions are available. No more attentions needed pay to build it from source. Download the official built libs from v1.8.1. I have used version 1.7.0 for Lite.AI now, you can downlod it from v1.7.0, but version 1.8.1 should also work, I guess ~ 🙃🤪🍀. For OpenCV, try to build from source(Linux) or down load the official built(Windows) from OpenCV 4.5.3. Then put the includes and libs into third_party directory of Lite.AI.

Clone the Lite.AI from sources:

git clone --depth=1 https://github.com/DefTruth/lite.ai.git  # latest

Build shared lib.

cd lite.ai
sh ./build.sh

cd ./build/lite.ai/lib && otool -L liblite.ai.0.0.1.dylib 
liblite.ai.0.0.1.dylib:
        @rpath/liblite.ai.0.0.1.dylib (compatibility version 0.0.1, current version 0.0.1)
        @rpath/libopencv_highgui.4.5.dylib (compatibility version 4.5.0, current version 4.5.2)
        @rpath/libonnxruntime.1.7.0.dylib (compatibility version 0.0.0, current version 1.7.0)
        ...

GPU Compatibility: See issue#10.

Expand for more details of How to link the shared lib of Lite.AI?

cd ../ && tree .
├── bin
├── include
│   ├── lite
│   │   ├── backend.h
│   │   ├── config.h
│   │   └── lite.h
│   └── ort
└── lib
    └── liblite.ai.0.0.1.dylib

Run the built examples:

cd ./build/lite.ai/bin && ls -lh | grep lite
-rwxr-xr-x  1 root  staff   301K Jun 26 23:10 liblite.ai.0.0.1.dylib
...
-rwxr-xr-x  1 root  staff   196K Jun 26 23:10 lite_yolov4
-rwxr-xr-x  1 root  staff   196K Jun 26 23:10 lite_yolov5
...

./lite_yolov5
LITEORT_DEBUG LogId: ../../../hub/onnx/cv/yolov5s.onnx
=============== Input-Dims ==============
...
detected num_anchors: 25200
generate_bboxes num: 66
Default Version Detected Boxes Num: 5

To link lite.ai shared lib. You need to make sure that OpenCV and onnxruntime are linked correctly. Just like:

cmake_minimum_required(VERSION 3.17)
project(testlite.ai)
set(CMAKE_CXX_STANDARD 11)
set(CMAKE_BUILD_TYPE debug)
# link opencv.
set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/opencv/lib/cmake/opencv4)
find_package(OpenCV 4 REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
# link onnxruntime.
set(ONNXRUNTIME_DIR ${CMAKE_SOURCE_DIR}/onnxruntime/)
set(ONNXRUNTIME_INCLUDE_DIR ${ONNXRUNTIME_DIR}/include)
set(ONNXRUNTIME_LIBRARY_DIR ${ONNXRUNTIME_DIR}/lib)
include_directories(${ONNXRUNTIME_INCLUDE_DIR})
link_directories(${ONNXRUNTIME_LIBRARY_DIR})
# link lite.ai.
set(LITEHUB_DIR ${CMAKE_SOURCE_DIR}/lite.ai)
set(LITEHUB_INCLUDE_DIR ${LITEHUB_DIR}/include)
set(LITEHUB_LIBRARY_DIR ${LITEHUB_DIR}/lib)
include_directories(${LITEHUB_INCLUDE_DIR})
link_directories(${LITEHUB_LIBRARY_DIR})
# add your executable
add_executable(lite_yolov5 test_lite_yolov5.cpp)
target_link_libraries(lite_yolov5 lite.ai onnxruntime ${OpenCV_LIBS})

A minimum example to show you how to link the shared lib of Lite.AI correctly for your own project can be found at lite.ai-release .

3. Model Zoo.

3.1 Namespace and Lite.AI modules.

Lite.AI contains 60+ AI models with 100+ frozen pretrained .onnx files now. They come from different fields of computer vision. Click the Expand ▶️ button for more details.

Expand Details for Namespace and Lite.AI modules.

Namepace	Details
lite::cv::detection	Object Detection. one-stage and anchor-free detectors, YoloV5, YoloV4, SSD, etc. ✅
lite::cv::classification	Image Classification. DensNet, ShuffleNet, ResNet, IBNNet, GhostNet, etc. ✅
lite::cv::faceid	Face Recognition. ArcFace, CosFace, CurricularFace, etc. ❇️
lite::cv::face	Face Analysis. detect, align, pose, attr, etc. ❇️
lite::cv::face::detect	Face Detection. UltraFace, RetinaFace, FaceBoxes, PyramidBox, etc. ❇️
lite::cv::face::align	Face Alignment. PFLD(106), FaceLandmark1000(1000 landmarks), PRNet, etc. ❇️
lite::cv::face::pose	Head Pose Estimation. FSANet, etc. ❇️
lite::cv::face::attr	Face Attributes. Emotion, Age, Gender. EmotionFerPlus, VGG16Age, etc. ❇️
lite::cv::segmentation	Object Segmentation. Such as FCN, DeepLabV3, etc. ⚠️
lite::cv::style	Style Transfer. Contains neural style transfer now, such as FastStyleTransfer. ⚠️
lite::cv::matting	Image Matting. Object and Human matting. ⚠️
lite::cv::colorization	Colorization. Make Gray image become RGB. ⚠️
lite::cv::resolution	Super Resolution. ⚠️

3.2 Lite.AI's Classes and Pretrained Files.

Correspondence between the classes in Lite.AI and pretrained model files can be found at lite.ai.hub.onnx.md. For examples, the pretrained model files for lite::cv::detection::YoloV5 and lite::cv::detection::YoloX are listed as follows.

Expand Examples for Lite.AI's Classes and Pretrained Files.

Class	Pretrained ONNX Files	Rename or Converted From (Repo)	Size
lite::cv::detection::YoloV5	yolov5l.onnx	yolov5 (🔥🔥💥↑)	188Mb
lite::cv::detection::YoloV5	yolov5m.onnx	yolov5 (🔥🔥💥↑)	85Mb
lite::cv::detection::YoloV5	yolov5s.onnx	yolov5 (🔥🔥💥↑)	29Mb
lite::cv::detection::YoloV5	yolov5x.onnx	yolov5 (🔥🔥💥↑)	351Mb
lite::cv::detection::YoloX	yolox_x.onnx	YOLOX (🔥🔥!!↑)	378Mb
lite::cv::detection::YoloX	yolox_l.onnx	YOLOX (🔥🔥!!↑)	207Mb
lite::cv::detection::YoloX	yolox_m.onnx	YOLOX (🔥🔥!!↑)	97Mb
lite::cv::detection::YoloX	yolox_s.onnx	YOLOX (🔥🔥!!↑)	34Mb
lite::cv::detection::YoloX	yolox_tiny.onnx	YOLOX (🔥🔥!!↑)	19Mb
lite::cv::detection::YoloX	yolox_nano.onnx	YOLOX (🔥🔥!!↑)	3.5Mb

It means that you can load the the any one yolov5*.onnx and yolox_*.onnx according to your application through the same Lite.AI classes, such as YoloV5, YoloX, etc.

auto *yolov5 = new lite::cv::detection::YoloV5("yolov5x.onnx");  // for server
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5l.onnx"); 
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5m.onnx");  
auto *yolov5 = new lite::cv::detection::YoloV5("yolov5s.onnx");  // for mobile device 
auto *yolox = new lite::cv::detection::YoloX("yolox_x.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_l.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_m.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_s.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_tiny.onnx");  
auto *yolox = new lite::cv::detection::YoloX("yolox_nano.onnx");  // 3.5Mb only !

3.3 Model Zoo for Lite.AI.

Note that the models here are all from third-party projects. Most of the models were converted by Lite.AI. In Lite.AI, different names of the same algorithm mean that the corresponding models come from different repositories, different implementations, or use different training data, etc. ✅ means passed the test and ⚠️ means not implements yet but coming soon. For classes which denoted ✅, you can use it through lite::cv::Type::Class syntax, such as lite::cv::detection::YoloV5 . More details can be found at Examples for Lite.AI .
(Baidu Drive code: 8gin)

Object Detection.

Class	Size	From	Awesome	Type	State	Usage
YoloV5	28M	yolov5	🔥🔥💥↑	detection	✅	demo
YoloV3	236M	onnx-models	🔥🔥🔥↑	detection	✅	demo
TinyYoloV3	33M	onnx-models	🔥🔥🔥↑	detection	✅	demo
YoloV4	176M	YOLOv4...	🔥🔥🔥↑	detection	✅	demo
SSD	76M	onnx-models	🔥🔥🔥↑	detection	✅	demo
SSDMobileNetV1	27M	onnx-models	🔥🔥🔥↑	detection	✅	demo
YoloX	3.5M	YOLOX	🔥🔥new↑	detection	✅	demo

Face Detection.

Class	Size	From	Awesome	Type	State	Usage
UltraFace	1.1M	Ultra-Light...	🔥🔥🔥↑	face::detect	✅	demo
RetinaFace	1.6M	...Retinaface	🔥🔥🔥↑	face::detect	✅	demo
FaceBoxes	3.8M	FaceBoxes	🔥🔥↑	face::detect	✅	demo

Face Alignment.

Class	Size	From	Awesome	Type	State	Usage
PFLD	1.0M	pfld_106_...	🔥🔥↑	face::align	✅	demo
PFLD98	4.8M	PFLD...	🔥🔥↑	face::align	✅️	demo
MobileNetV268	9.4M	...landmark	🔥🔥↑	face::align	✅️️	demo
MobileNetV2SE68	11M	...landmark	🔥🔥↑	face::align	✅️️	demo
PFLD68	2.8M	...landmark	🔥🔥↑	face::align	✅️	demo
FaceLandmark1000	2.0M	FaceLandm...	🔥↑	face::align	✅️	demo

Face Recognition.

Class	Size	From	Awesome	Type	State	Usage
GlintArcFace	92M	insightface	🔥🔥🔥↑	faceid	✅	demo
GlintCosFace	92M	insightface	🔥🔥🔥↑	faceid	✅	demo
GlintPartialFC	170M	insightface	🔥🔥🔥↑	faceid	✅	demo
FaceNet	89M	facenet...	🔥🔥🔥↑	faceid	✅	demo
FocalArcFace	166M	face.evoLVe...	🔥🔥🔥↑	faceid	✅	demo
FocalAsiaArcFace	166M	face.evoLVe...	🔥🔥🔥↑	faceid	✅	demo
TencentCurricularFace	249M	TFace	🔥🔥↑	faceid	✅	demo
TencentCifpFace	130M	TFace	🔥🔥↑	faceid	✅	demo
CenterLossFace	280M	center-loss...	🔥🔥↑	faceid	✅	demo
SphereFace	80M	sphere...	🔥🔥↑	faceid	✅️	demo
PoseRobustFace	92M	DREAM	🔥🔥↑	faceid	✅️	demo
NaivePoseRobustFace	43M	DREAM	🔥🔥↑	faceid	✅️	demo
MobileFaceNet	3.8M	MobileFace...	🔥🔥↑	faceid	✅	demo
CavaGhostArcFace	15M	cavaface...	🔥🔥↑	faceid	✅	demo
CavaCombinedFace	250M	cavaface...	🔥🔥↑	faceid	✅	demo
MobileSEFocalFace	4.5M	face_recog...	🔥🔥↑	faceid	✅	demo

⚠️ Expand More Details for Lite.AI's Model Zoo.

Head Pose Estimation.

Class	Size	From	Awesome	File	Type	State	Usage
FSANet	1.2M	...fsanet...	🔥↑		face::pose	✅	demo

Face Attributes.

Class	Size	From	Awesome	Type	State	Usage
AgeGoogleNet	23M	onnx-models	🔥🔥🔥↑	face::attr	✅	demo
GenderGoogleNet	23M	onnx-models	🔥🔥🔥↑	face::attr	✅	demo
EmotionFerPlus	33M	onnx-models	🔥🔥🔥↑	face::attr	✅	demo
VGG16Age	514M	onnx-models	🔥🔥🔥↑	face::attr	✅	demo
VGG16Gender	512M	onnx-models	🔥🔥🔥↑	face::attr	✅	demo
SSRNet	190K	SSR_Net...	🔥↑	face::attr	✅	demo
EfficientEmotion7	15M	face-emo...	🔥↑	face::attr	✅️	demo
EfficientEmotion8	15M	face-emo...	🔥↑	face::attr	✅	demo
MobileEmotion7	13M	face-emo...	🔥↑	face::attr	✅	demo
ReXNetEmotion7	30M	face-emo...	🔥↑	face::attr	✅	demo

Classification.

Class	Size	From	Awesome	Type	State	Usage
EfficientNetLite4	49M	onnx-models	🔥🔥🔥↑	classification	✅	demo
ShuffleNetV2	8.7M	onnx-models	🔥🔥🔥↑	classification	✅	demo
DenseNet121	30.7M	torchvision	🔥🔥🔥↑	classification	✅	demo
GhostNet	20M	torchvision	🔥🔥🔥↑	classification	✅	demo
HdrDNet	13M	torchvision	🔥🔥🔥↑	classification	✅	demo
IBNNet	97M	torchvision	🔥🔥🔥↑	classification	✅	demo
MobileNetV2	13M	torchvision	🔥🔥🔥↑	classification	✅	demo
ResNet	44M	torchvision	🔥🔥🔥↑	classification	✅	demo
ResNeXt	95M	torchvision	🔥🔥🔥↑	classification	✅	demo

Segmentation.

Class	Size	From	Awesome	File	Type	State	Usage
DeepLabV3ResNet101	232M	torchvision	🔥🔥🔥↑		segmentation	✅	demo
FCNResNet101	207M	torchvision	🔥🔥🔥↑		segmentation	✅	demo

Style Transfer.

Class	Size	From	Awesome	File	Type	State	Usage
FastStyleTransfer	6.4M	onnx-models	🔥🔥🔥↑		style	✅	demo

Colorization.

Class	Size	From	Awesome	File	Type	State	Usage
Colorizer	123M	colorization	🔥🔥🔥↑		colorization	✅	demo

Super Resolution.

Class	Size	From	Awesome	File	Type	State	Usage
SubPixelCNN	234K	...PIXEL...	🔥↑		resolution	✅	demo

4. Examples for Lite.AI.

More examples can be found at lite.ai-demos. Note that the default backend for Lite.AI is onnxruntime, for the reason that onnxruntime supports the most of onnx's operators. Click the Expand ▶️ button will show you more examples for the specific topic you are interested in.

Example0: Object Detection using YoloV5. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/yolov5s.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_yolov5_1.jpg";
  std::string save_img_path = "../../../logs/test_lite_yolov5_1.jpg";

  auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path); 
  std::vector<lite::cv::types::Boxf> detected_boxes;
  cv::Mat img_bgr = cv::imread(test_img_path);
  yolov5->detect(img_bgr, detected_boxes);
  
  lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
  cv::imwrite(save_img_path, img_bgr);  
  
  delete yolov5;
}

The output is:

Or you can use Newest 🔥🔥 ! YOLO series's detector YOLOX . They got the similar results.

Example1: 1000 Facial Landmarks Detection using FaceLandmarks1000. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/FaceLandmark1000.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_face_landmarks_0.png";
  std::string save_img_path = "../../../logs/test_lite_face_landmarks_1000.jpg";
    
  auto *face_landmarks_1000 = new lite::cv::face::align::FaceLandmark1000(onnx_path);

  lite::cv::types::Landmarks landmarks;
  cv::Mat img_bgr = cv::imread(test_img_path);
  face_landmarks_1000->detect(img_bgr, landmarks);
  lite::cv::utils::draw_landmarks_inplace(img_bgr, landmarks);
  cv::imwrite(save_img_path, img_bgr);
  
  delete face_landmarks_1000;
}

The output is:

Example2: Colorization using colorization. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/eccv16-colorizer.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_colorizer_1.jpg";
  std::string save_img_path = "../../../logs/test_lite_eccv16_colorizer_1.jpg";
  
  auto *colorizer = new lite::cv::colorization::Colorizer(onnx_path);
  
  cv::Mat img_bgr = cv::imread(test_img_path);
  lite::cv::types::ColorizeContent colorize_content;
  colorizer->detect(img_bgr, colorize_content);
  
  if (colorize_content.flag) cv::imwrite(save_img_path, colorize_content.mat);
  delete colorizer;
}

The output is:

Example3: Face Recognition using ArcFace. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ms1mv3_arcface_r100.onnx";
  std::string test_img_path0 = "../../../examples/lite/resources/test_lite_faceid_0.png";
  std::string test_img_path1 = "../../../examples/lite/resources/test_lite_faceid_1.png";
  std::string test_img_path2 = "../../../examples/lite/resources/test_lite_faceid_2.png";

  auto *glint_arcface = new lite::cv::faceid::GlintArcFace(onnx_path);

  lite::cv::types::FaceContent face_content0, face_content1, face_content2;
  cv::Mat img_bgr0 = cv::imread(test_img_path0);
  cv::Mat img_bgr1 = cv::imread(test_img_path1);
  cv::Mat img_bgr2 = cv::imread(test_img_path2);
  glint_arcface->detect(img_bgr0, face_content0);
  glint_arcface->detect(img_bgr1, face_content1);
  glint_arcface->detect(img_bgr2, face_content2);

  if (face_content0.flag && face_content1.flag && face_content2.flag)
  {
    float sim01 = lite::cv::utils::math::cosine_similarity<float>(
        face_content0.embedding, face_content1.embedding);
    float sim02 = lite::cv::utils::math::cosine_similarity<float>(
        face_content0.embedding, face_content2.embedding);
    std::cout << "Detected Sim01: " << sim  << " Sim02: " << sim02 << std::endl;
  }

  delete glint_arcface;
}

The output is:

Detected Sim01: 0.721159 Sim02: -0.0626267

Example4: Face Detection using UltraFace. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ultraface-rfb-640.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_ultraface.jpg";
  std::string save_img_path = "../../../logs/test_lite_ultraface.jpg";

  auto *ultraface = new lite::cv::face::detect::UltraFace(onnx_path);

  std::vector<lite::cv::types::Boxf> detected_boxes;
  cv::Mat img_bgr = cv::imread(test_img_path);
  ultraface->detect(img_bgr, detected_boxes);
  lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
  cv::imwrite(save_img_path, img_bgr);

  delete ultraface;
}

The output is:

⚠️ Expand All Examples for Each Topic in Lite.AI.

4.1 Expand Examples for Object Detection.

4.1 Object Detection using YoloV5. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
std::string onnx_path = "../../../hub/onnx/cv/yolov5s.onnx";
std::string test_img_path = "../../../examples/lite/resources/test_lite_yolov5_1.jpg";
std::string save_img_path = "../../../logs/test_lite_yolov5_1.jpg";

auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path);
std::vector<lite::cv::types::Boxf> detected_boxes;
cv::Mat img_bgr = cv::imread(test_img_path);
yolov5->detect(img_bgr, detected_boxes);

lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
cv::imwrite(save_img_path, img_bgr);

delete yolov5;
}

The output is:

Or you can use Newest 🔥🔥 ! YOLO series's detector YOLOX . They got the similar results.

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/yolox_s.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_yolox_1.jpg";
  std::string save_img_path = "../../../logs/test_lite_yolox_1.jpg";

  auto *yolox = new lite::cv::detection::YoloX(onnx_path); 
  std::vector<lite::cv::types::Boxf> detected_boxes;
  cv::Mat img_bgr = cv::imread(test_img_path);
  yolox->detect(img_bgr, detected_boxes);
  
  lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
  cv::imwrite(save_img_path, img_bgr);  
  
  delete yolox;
}

The output is:

More classes for general object detection.

auto *detector = new lite::cv::detection::YoloX(onnx_path); // new !!!
auto *detector = new lite::cv::detection::YoloV4(onnx_path); 
auto *detector = new lite::cv::detection::YoloV3(onnx_path); 
auto *detector = new lite::cv::detection::TinyYoloV3(onnx_path); 
auto *detector = new lite::cv::detection::SSD(onnx_path); 
auto *detector = new lite::cv::detection::SSDMobileNetV1(onnx_path);

4.2 Expand Examples for Face Recognition.

4.2 Face Recognition using ArcFace. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ms1mv3_arcface_r100.onnx";
  std::string test_img_path0 = "../../../examples/lite/resources/test_lite_faceid_0.png";
  std::string test_img_path1 = "../../../examples/lite/resources/test_lite_faceid_1.png";
  std::string test_img_path2 = "../../../examples/lite/resources/test_lite_faceid_2.png";

  auto *glint_arcface = new lite::cv::faceid::GlintArcFace(onnx_path);

  lite::cv::types::FaceContent face_content0, face_content1, face_content2;
  cv::Mat img_bgr0 = cv::imread(test_img_path0);
  cv::Mat img_bgr1 = cv::imread(test_img_path1);
  cv::Mat img_bgr2 = cv::imread(test_img_path2);
  glint_arcface->detect(img_bgr0, face_content0);
  glint_arcface->detect(img_bgr1, face_content1);
  glint_arcface->detect(img_bgr2, face_content2);

  if (face_content0.flag && face_content1.flag && face_content2.flag)
  {
    float sim01 = lite::cv::utils::math::cosine_similarity<float>(
        face_content0.embedding, face_content1.embedding);
    float sim02 = lite::cv::utils::math::cosine_similarity<float>(
        face_content0.embedding, face_content2.embedding);
    std::cout << "Detected Sim01: " << sim  << " Sim02: " << sim02 << std::endl;
  }

  delete glint_arcface;
}

The output is:

Detected Sim01: 0.721159 Sim02: -0.0626267

More classes for face recognition.

auto *recognition = new lite::cv::faceid::GlintCosFace(onnx_path);  // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::GlintArcFace(onnx_path);  // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::GlintPartialFC(onnx_path); // DeepGlint(insightface)
auto *recognition = new lite::cv::faceid::FaceNet(onnx_path);
auto *recognition = new lite::cv::faceid::FocalArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::FocalAsiaArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::TencentCurricularFace(onnx_path); // Tencent(TFace)
auto *recognition = new lite::cv::faceid::TencentCifpFace(onnx_path); // Tencent(TFace)
auto *recognition = new lite::cv::faceid::CenterLossFace(onnx_path);
auto *recognition = new lite::cv::faceid::SphereFace(onnx_path);
auto *recognition = new lite::cv::faceid::PoseRobustFace(onnx_path);
auto *recognition = new lite::cv::faceid::NaivePoseRobustFace(onnx_path);
auto *recognition = new lite::cv::faceid::MobileFaceNet(onnx_path); // 3.8Mb only !
auto *recognition = new lite::cv::faceid::CavaGhostArcFace(onnx_path);
auto *recognition = new lite::cv::faceid::CavaCombinedFace(onnx_path);
auto *recognition = new lite::cv::faceid::MobileSEFocalFace(onnx_path); // 4.5Mb only !

4.3 Expand Examples for Segmentation.

4.3 Segmentation using DeepLabV3ResNet101. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/deeplabv3_resnet101_coco.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_deeplabv3_resnet101.png";
  std::string save_img_path = "../../../logs/test_lite_deeplabv3_resnet101.jpg";

  auto *deeplabv3_resnet101 = new lite::cv::segmentation::DeepLabV3ResNet101(onnx_path, 16); // 16 threads

  lite::cv::types::SegmentContent content;
  cv::Mat img_bgr = cv::imread(test_img_path);
  deeplabv3_resnet101->detect(img_bgr, content);

  if (content.flag)
  {
    cv::Mat out_img;
    cv::addWeighted(img_bgr, 0.2, content.color_mat, 0.8, 0., out_img);
    cv::imwrite(save_img_path, out_img);
    if (!content.names_map.empty())
    {
      for (auto it = content.names_map.begin(); it != content.names_map.end(); ++it)
      {
        std::cout << it->first << " Name: " << it->second << std::endl;
      }
    }
  }
  delete deeplabv3_resnet101;
}

The output is:

More classes for segmentation.

auto *segment = new lite::cv::segmentation::FCNResNet101(onnx_path);

4.4 Expand Examples for Face Attributes Analysis.

4.4 Age Estimation using SSRNet . Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ssrnet.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_ssrnet.jpg";
  std::string save_img_path = "../../../logs/test_lite_ssrnet.jpg";

  lite::cv::face::attr::SSRNet *ssrnet = new lite::cv::face::attr::SSRNet(onnx_path);

  lite::cv::types::Age age;
  cv::Mat img_bgr = cv::imread(test_img_path);
  ssrnet->detect(img_bgr, age);
  lite::cv::utils::draw_age_inplace(img_bgr, age);
  cv::imwrite(save_img_path, img_bgr);
  std::cout << "Default Version Done! Detected SSRNet Age: " << age.age << std::endl;

  delete ssrnet;
}

The output is:

More classes for face attributes analysis.

auto *attribute = new lite::cv::face::attr::AgeGoogleNet(onnx_path);  
auto *attribute = new lite::cv::face::attr::GenderGoogleNet(onnx_path); 
auto *attribute = new lite::cv::face::attr::EmotionFerPlus(onnx_path);
auto *attribute = new lite::cv::face::attr::VGG16Age(onnx_path);
auto *attribute = new lite::cv::face::attr::VGG16Gender(onnx_path);
auto *attribute = new lite::cv::face::attr::EfficientEmotion7(onnx_path); // 7 emotions, 15Mb only!
auto *attribute = new lite::cv::face::attr::EfficientEmotion8(onnx_path); // 8 emotions, 15Mb only!
auto *attribute = new lite::cv::face::attr::MobileEmotion7(onnx_path); // 7 emotions
auto *attribute = new lite::cv::face::attr::ReXNetEmotion7(onnx_path); // 7 emotions

4.5 Expand Examples for Image Classification.

4.5 1000 Classes Classification using DenseNet. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/densenet121.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_densenet.jpg";

  auto *densenet = new lite::cv::classification::DenseNet(onnx_path);

  lite::cv::types::ImageNetContent content;
  cv::Mat img_bgr = cv::imread(test_img_path);
  densenet->detect(img_bgr, content);
  if (content.flag)
  {
    const unsigned int top_k = content.scores.size();
    if (top_k > 0)
    {
      for (unsigned int i = 0; i < top_k; ++i)
        std::cout << i + 1
                  << ": " << content.labels.at(i)
                  << ": " << content.texts.at(i)
                  << ": " << content.scores.at(i)
                  << std::endl;
    }
  }
  delete densenet;
}

The output is:

More classes for image classification.

auto *classifier = new lite::cv::classification::EfficientNetLite4(onnx_path);  
auto *classifier = new lite::cv::classification::ShuffleNetV2(onnx_path); 
auto *classifier = new lite::cv::classification::GhostNet(onnx_path);
auto *classifier = new lite::cv::classification::HdrDNet(onnx_path);
auto *classifier = new lite::cv::classification::IBNNet(onnx_path);
auto *classifier = new lite::cv::classification::MobileNetV2(onnx_path); 
auto *classifier = new lite::cv::classification::ResNet(onnx_path); 
auto *classifier = new lite::cv::classification::ResNeXt(onnx_path);

4.6 Expand Examples for Face Detection.

4.6 Face Detection using UltraFace. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/ultraface-rfb-640.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_ultraface.jpg";
  std::string save_img_path = "../../../logs/test_lite_ultraface.jpg";

  auto *ultraface = new lite::cv::face::detect::UltraFace(onnx_path);

  std::vector<lite::cv::types::Boxf> detected_boxes;
  cv::Mat img_bgr = cv::imread(test_img_path);
  ultraface->detect(img_bgr, detected_boxes);
  lite::cv::utils::draw_boxes_inplace(img_bgr, detected_boxes);
  cv::imwrite(save_img_path, img_bgr);

  delete ultraface;
}

The output is:

4.7 Expand Examples for Colorization.

4.7 Colorization using colorization. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/eccv16-colorizer.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_colorizer_1.jpg";
  std::string save_img_path = "../../../logs/test_lite_eccv16_colorizer_1.jpg";
  
  auto *colorizer = new lite::cv::colorization::Colorizer(onnx_path);
  
  cv::Mat img_bgr = cv::imread(test_img_path);
  lite::cv::types::ColorizeContent colorize_content;
  colorizer->detect(img_bgr, colorize_content);
  
  if (colorize_content.flag) cv::imwrite(save_img_path, colorize_content.mat);
  delete colorizer;
}

The output is:

4.8 Expand Examples for Head Pose Estimation.

4.8 Head Pose Estimation using FSANet. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/fsanet-var.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_fsanet.jpg";
  std::string save_img_path = "../../../logs/test_lite_fsanet.jpg";

  auto *fsanet = new lite::cv::face::pose::FSANet(onnx_path);
  cv::Mat img_bgr = cv::imread(test_img_path);
  lite::cv::types::EulerAngles euler_angles;
  fsanet->detect(img_bgr, euler_angles);
  
  if (euler_angles.flag)
  {
    lite::cv::utils::draw_axis_inplace(img_bgr, euler_angles);
    cv::imwrite(save_img_path, img_bgr);
    std::cout << "yaw:" << euler_angles.yaw << " pitch:" << euler_angles.pitch << " row:" << euler_angles.roll << std::endl;
  }
  delete fsanet;
}

The output is:

4.9 Expand Examples for Face Alignment.

4.9 1000 Facial Landmarks Detection using FaceLandmarks1000. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/FaceLandmark1000.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_face_landmarks_0.png";
  std::string save_img_path = "../../../logs/test_lite_face_landmarks_1000.jpg";
    
  auto *face_landmarks_1000 = new lite::cv::face::align::FaceLandmark1000(onnx_path);

  lite::cv::types::Landmarks landmarks;
  cv::Mat img_bgr = cv::imread(test_img_path);
  face_landmarks_1000->detect(img_bgr, landmarks);
  lite::cv::utils::draw_landmarks_inplace(img_bgr, landmarks);
  cv::imwrite(save_img_path, img_bgr);
  
  delete face_landmarks_1000;
}

The output is:

More classes for face alignment.

auto *align = new lite::cv::face::align::PFLD(onnx_path);  // 106 landmarks
auto *align = new lite::cv::face::align::PFLD98(onnx_path);  // 98 landmarks
auto *align = new lite::cv::face::align::PFLD68(onnx_path);  // 68 landmarks
auto *align = new lite::cv::face::align::MobileNetV268(onnx_path);  // 68 landmarks
auto *align = new lite::cv::face::align::MobileNetV2SE68(onnx_path);  // 68 landmarks
auto *align = new lite::cv::face::align::FaceLandmark1000(onnx_path);  // 1000 landmarks !

4.10 Expand Examples for Style Transfer.

4.10 Style Transfer using FastStyleTransfer. Download model from Model-Zoo².

#include "lite/lite.h"

static void test_default()
{
  std::string onnx_path = "../../../hub/onnx/cv/style-candy-8.onnx";
  std::string test_img_path = "../../../examples/lite/resources/test_lite_fast_style_transfer.jpg";
  std::string save_img_path = "../../../logs/test_lite_fast_style_transfer_candy.jpg";
  
  auto *fast_style_transfer = new lite::cv::style::FastStyleTransfer(onnx_path);
 
  lite::cv::types::StyleContent style_content;
  cv::Mat img_bgr = cv::imread(test_img_path);
  fast_style_transfer->detect(img_bgr, style_content);

  if (style_content.flag) cv::imwrite(save_img_path, style_content.mat);
  delete fast_style_transfer;
}

The output is:

4.11 Expand Examples for Image Matting.

todo⚠️

5. Lite.AI API Docs.

5.1 Default Version APIs.

More details of Default Version APIs can be found at default-version-api-docs . For examples, the interface for YoloV5 is:

lite::cv::detection::YoloV5

void detect(const cv::Mat &mat, std::vector<types::Boxf> &detected_boxes, 
            float score_threshold = 0.25f, float iou_threshold = 0.45f,
            unsigned int topk = 100, unsigned int nms_type = NMS::OFFSET);

Expand for ONNXRuntime, MNN and NCNN version APIs.

5.2 ONNXRuntime Version APIs.

More details of ONNXRuntime Version APIs can be found at onnxruntime-version-api-docs . For examples, the interface for YoloV5 is:

lite::onnxruntime::cv::detection::YoloV5

void detect(const cv::Mat &mat, std::vector<types::Boxf> &detected_boxes, 
            float score_threshold = 0.25f, float iou_threshold = 0.45f,
            unsigned int topk = 100, unsigned int nms_type = NMS::OFFSET);

5.3 MNN Version APIs.

(todo⚠️: Not implementation now, coming soon.)

lite::mnn::cv::detection::YoloV5

lite::mnn::cv::detection::YoloV4

lite::mnn::cv::detection::YoloV3

lite::mnn::cv::detection::SSD

...

5.4 NCNN Version APIs.

(todo⚠️: Not implementation now, coming soon.)

lite::ncnn::cv::detection::YoloV5

lite::ncnn::cv::detection::YoloV4

lite::ncnn::cv::detection::YoloV3

lite::ncnn::cv::detection::SSD

...

6. Other Docs.

Expand More Details for Other Docs.

6.1 Docs for ONNXRuntime.

6.2 Docs for third_party.

Other build documents for different engines and different targets will be added later.

Library	Target	Docs
OpenCV	mac-x86_64	opencv-mac-x86_64-build.zh.md
OpenCV	android-arm	opencv-static-android-arm-build.zh.md
onnxruntime	mac-x86_64	onnxruntime-mac-x86_64-build.zh.md
onnxruntime	android-arm	onnxruntime-android-arm-build.zh.md
NCNN	mac-x86_64	todo⚠️
MNN	mac-x86_64	todo⚠️
TNN	mac-x86_64	todo⚠️

7. References.

Many thanks to the following projects. All the Lite.AI's models are sourced from these repos.

YOLOX (🔥🔥new!!↑)
insightface (🔥🔥🔥↑)
yolov5 (🔥🔥💥↑)

Expand More Details for References.

headpose-fsanet-pytorch (🔥↑)
pfld_106_face_landmarks (🔥🔥↑)
Ultra-Light-Fast-Generic-Face-Detector-1MB (🔥🔥🔥↑)
onnx-models (🔥🔥🔥↑)
SSR_Net_Pytorch (🔥↑)
colorization (🔥🔥🔥↑)
SUB_PIXEL_CNN (🔥↑)
YOLOv4-pytorch (🔥🔥🔥↑)
torchvision (🔥🔥🔥↑)
facenet-pytorch (🔥↑)
face.evoLVe.PyTorch (🔥🔥🔥↑)
TFace (🔥🔥↑)
center-loss.pytorch (🔥🔥↑)
sphereface_pytorch (🔥🔥↑)
DREAM (🔥🔥↑)
MobileFaceNet_Pytorch (🔥🔥↑)
cavaface.pytorch (🔥🔥↑)
CurricularFace (🔥🔥↑)
face-emotion-recognition (🔥↑)
face_recognition.pytorch (🔥🔥↑)
PFLD-pytorch (🔥🔥↑)
pytorch_face_landmark (🔥🔥↑)
FaceLandmark1000 (🔥🔥↑)
Pytorch_Retinaface (🔥🔥🔥↑)
FaceBoxes (🔥🔥↑)

Star 🌟👆🏻 this repo if it does any helps to you ~

License.

Only the code of Lite.AI is released under the MIT License.

Citations.

If you use this library in your project, please, cite it as follows.

@code{lite.ai2021,
  title={Lite.AI: A simple and user friendly C++ library of awesome AI models.},
  url={https://github.com/DefTruth/lite.ai},
  note={Open-source software available at https://github.com/DefTruth/lite.ai},
  author={Qiu},
  year={2021}
}

Name		Name	Last commit message	Last commit date
Latest commit History 464 Commits
build		build
cmake		cmake
docs		docs
examples/lite		examples/lite
hub/onnx/cv		hub/onnx/cv
lite		lite
logs		logs
mnn		mnn
ncnn		ncnn
ort		ort
third_party		third_party
.gitignore		.gitignore
CMakeLists.txt		CMakeLists.txt
LICENSE		LICENSE
README.md		README.md
build.sh		build.sh
requirements.txt		requirements.txt

License

reed4u/lite.ai

Folders and files

Latest commit

History

Repository files navigation

Lite.AI 🚀🚀🌟

Introduction.

Important Notes !!!

More Notes !!!

Working Notes. 👇🏻

Contents.

1. Dependencies.

Mac OS.

Linux.

Windows.

Inference Engine Plans:

2. Build Lite.AI.

Linux and Windows.

3. Model Zoo.

3.1 Namespace and Lite.AI modules.

3.2 Lite.AI's Classes and Pretrained Files.

3.3 Model Zoo for Lite.AI.

4. Examples for Lite.AI.

Example0: Object Detection using YoloV5. Download model from Model-Zoo2.

Example1: 1000 Facial Landmarks Detection using FaceLandmarks1000. Download model from Model-Zoo2.

Example2: Colorization using colorization. Download model from Model-Zoo2.

Example3: Face Recognition using ArcFace. Download model from Model-Zoo2.

Example4: Face Detection using UltraFace. Download model from Model-Zoo2.

4.1 Object Detection using YoloV5. Download model from Model-Zoo2.

4.2 Face Recognition using ArcFace. Download model from Model-Zoo2.

4.3 Segmentation using DeepLabV3ResNet101. Download model from Model-Zoo2.

4.4 Age Estimation using SSRNet . Download model from Model-Zoo2.

4.5 1000 Classes Classification using DenseNet. Download model from Model-Zoo2.

4.6 Face Detection using UltraFace. Download model from Model-Zoo2.

4.7 Colorization using colorization. Download model from Model-Zoo2.

4.8 Head Pose Estimation using FSANet. Download model from Model-Zoo2.

4.9 1000 Facial Landmarks Detection using FaceLandmarks1000. Download model from Model-Zoo2.

4.10 Style Transfer using FastStyleTransfer. Download model from Model-Zoo2.

5. Lite.AI API Docs.

5.1 Default Version APIs.

5.2 ONNXRuntime Version APIs.

5.3 MNN Version APIs.

5.4 NCNN Version APIs.

6. Other Docs.

6.1 Docs for ONNXRuntime.

6.2 Docs for third_party.

7. References.

License.

Citations.

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Example0: Object Detection using YoloV5. Download model from Model-Zoo².

Example1: 1000 Facial Landmarks Detection using FaceLandmarks1000. Download model from Model-Zoo².

Example2: Colorization using colorization. Download model from Model-Zoo².

Example3: Face Recognition using ArcFace. Download model from Model-Zoo².

Example4: Face Detection using UltraFace. Download model from Model-Zoo².

4.1 Object Detection using YoloV5. Download model from Model-Zoo².

4.2 Face Recognition using ArcFace. Download model from Model-Zoo².

4.3 Segmentation using DeepLabV3ResNet101. Download model from Model-Zoo².

4.4 Age Estimation using SSRNet . Download model from Model-Zoo².

4.5 1000 Classes Classification using DenseNet. Download model from Model-Zoo².

4.6 Face Detection using UltraFace. Download model from Model-Zoo².

4.7 Colorization using colorization. Download model from Model-Zoo².

4.8 Head Pose Estimation using FSANet. Download model from Model-Zoo².

4.9 1000 Facial Landmarks Detection using FaceLandmarks1000. Download model from Model-Zoo².

4.10 Style Transfer using FastStyleTransfer. Download model from Model-Zoo².

Packages