ibug.face_alignment

2D facial landmark detector based on FAN [1] with some pretrained weights. Our training code is available in this repostory: https://github.com/hhj1897/fan_training.

Prerequisites

• Numpy: $pip3 install numpy
• OpenCV: $pip3 install opencv-python
• PyTorch: $pip3 install torch torchvision
• ibug.face_detection (only needed by the test script): See this repository for details: https://github.com/hhj1897/face_detection.

How to Install

git clone https://github.com/hhj1897/face_alignment.git
cd face_alignment
pip install -e .

How to Test

• To test on live video: python face_alignment_test.py [-i webcam_index]
• To test on a video file: python face_alignment_test.py [-i input_file] [-o output_file]

How to Use

# Import the libraries
import cv2
from ibug.face_detection import RetinaFacePredictor
from ibug.face_alignment import FANPredictor
from ibug.face_alignment.utils import plot_landmarks

# Create a RetinaFace detector using Resnet50 backbone, with the confidence
# threshold set to 0.8
face_detector = RetinaFacePredictor(
    threshold=0.8, device='cuda:0',
    model=RetinaFacePredictor.get_model('resnet50'))

# Create a facial landmark detector
landmark_detector = FANPredictor(
    device='cuda:0', model=FANPredictor.get_model('2dfan2_alt'))

# Load a test image. Note that images loaded by OpenCV adopt the B-G-R channel
# order.
image = cv2.imread('test.png')

# Detect faces from the image
detected_faces = face_detector(image, rgb=False)

# Detect landmarks from the faces
# Note:
#   1. The input image must be a byte array of dimension HxWx3.
#   2. The input face boxes must be a array of dimension Nx4, N being the
#      number of faces. More columns are allowed, but only the first 4
#      columns will be used (which should be the left, top, right, and
#      bottom coordinates of the face).
#   3. The returned landmarks are stored in a Nx68x2 arrays, each row giving
#      the X and Y coordinates of a landmark.
#   4. The returned scores are stored in a Nx68 array. Scores are usually
#      within the range of 0 to 1, but could go slightly beyond.
landmarks, scores = landmark_detector(image, detected_faces, rgb=False)

# Draw the landmarks onto the image
for lmks, scs in zip(landmarks, scores):
    plot_landmarks(image, lmks, scs, threshold=0.2)

References

[1] Bulat, Adrian, and Georgios Tzimiropoulos. "How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks)." In Proceedings of the IEEE International Conference on Computer Vision, pp. 1021-1030. 2017.