The Head Gesture Detection (HGD) library provides a pre-trained model and a simple inference API for detecting head gestures in short videos. Under the hood, it uses Google MediaPipe for collecting landmark features required by the model.
Tested for Python 3.8, 3.9, and 3.10.
The best way to install HGD with its dependencies is from PyPI:
python3 -m pip install --upgrade hgd
Alternatively, to obtain the latest version from this repository:
git clone git@github.com:bhky/head-gesture-detection.git
cd head-gesture-detection
python3 -m pip install .
An easy way to try this library and the pre-trained model is to make a short video with your head gesture.
The code snippet below will perform the following:
- Start webcam.
- Collect the needed number of frames (default
60
) for the model. - End webcam automatically (or you can press
q
to end earlier). - Make prediction of your head gesture and print the result to STDOUT.
from hgd.inference import predict_video
# By default, the following call will download the pre-trained model weights
# and start your webcam.
result = predict_video()
print(result)
Alternatively, you could provide a pre-recorded video file:
from hgd.inference import predict_video
result = predict_video(
"your_head_gesture_video.mp4",
from_beginning=False,
motion_threshold=0.5, # Optionally tune the thresholds.
gesture_threshold=0.9
)
# The `from_beginning` flag controls whether the needed frames will be obtained
# from the beginning or toward the end of the video.
# Thresholds can be adjusted as needed, see explanation below.
The result is returned as a Python dictionary.
{
'gesture': 'turning',
'probabilities': {
'has_motion': 1.0,
'gestures': {
'nodding': 0.009188028052449226,
'turning': 0.9908120036125183
}
}
}
The following gesture
types are available:
nodding
- Repeatedly tilt your head upward and downward.turning
- Repeatedly turn your head leftward and rightward.stationary
- Not tilting or turning your head; translation motion is still treated as stationary.undefined
- Unrecognised gesture.
To determine the final gesture
:
- If
has_motion
probability is smaller thanmotion_threshold
(default0.5
),gesture
isstationary
. Other probabilities are irrelevant. - Otherwise, we will look for the largest probability from
gestures
:- If it is smaller than
gesture_threshold
(default0.9
),gesture
isundefined
, - else, the corresponding gesture label is selected (e.g.,
nodding
).
- If it is smaller than