Employing Fusion of Learned and Handcrafted Features for Unconstrained Ear Recognition

Models and demo code for http://digital-library.theiet.org/content/journals/10.1049/iet-bmt.2017.0210 (arXiv: http://arxiv.org/abs/1710.07662). This product is covered by U.S. Patent No. 10,423,823.

Authors

• Earnest E. Hansley
• Mauricio Pamplona Segundo
• Sudeep Sarkar

Models

• Side classifier (75MB): https://drive.google.com/file/d/1KYS_3tUWlxY7Gv5SjymSsnIUG-b8wApE/view?usp=sharing
• Landmark detector (1st stage, 75MB): https://drive.google.com/file/d/1OLCUhbY4lLgEUbC6dBdw4FaunQtkc1Ub/view?usp=sharing
• Landmark detector (2nd stage, 75MB): https://drive.google.com/file/d/15x9BPg7r5sk0ab7wAKd4h0Mo9DdrmM5F/view?usp=sharing
• CNN descriptor (71MB): https://drive.google.com/file/d/1Sg4oRUotuwheCkSKIesu0K7fWHzxsitl/view?usp=sharing

Requirements

This code only requires the master branch of OpenCV to work. You can install it locally with the following sequence of steps without making any modifications in a different OpenCV version previously installed in your system (Ubuntu 16.04):

$ sudo apt-get install build-essential
$ sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
$ sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
$ cd ~/your_choice/
$ git clone https://github.com/opencv/opencv.git
$ git clone https://github.com/opencv/opencv_contrib.git
$ mkdir opencv_install
$ cd opencv
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=../../opencv_install/ -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules/ ..
$ make -j 4
$ make install

Compiling and running

How to compile and run using a local OpenCV installation:

$ g++ -std=c++11 demo.cpp -o demo `pkg-config --libs --cflags ~/your_choice/opencv_install/lib/pkgconfig/opencv.pc`
$ export LD_LIBRARY_PATH=~/your_choice/opencv_install/lib/
$ ./demo your_image.{png,jpg,...}

What this demo does?

(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)

1. Load an input image of a cropped ear
2. Convert image in step #1 to grayscale
3. Normalize the image in step #2 using a coarse parameter estimation
4. Run side classification and flip the image in step #3 horizontally if necessary
5. Detect landmarks in the image in step #4 using 1st stage, which is more robust to pose variations
6. Adjust parameters using landmarks obtained in step #5 and renormalize the image in step #2
7. Detect landmarks in the image of step #6 using 2nd stage, which is more accurate for well normalized images
8. Adjust parameters using landmarks obtained in step #7 and renormalize the image in step #2, then extract a CNN descriptor (output.txt) from the normalized image (output.png)

Citing

If you find the code in this repository useful in your research, please consider citing:

@ARTICLE{
   iet:/content/journals/10.1049/iet-bmt.2017.0210,
   author = {Earnest Hansley},
   author = {Mauricio Pamplona Segundo},
   author = {Sudeep Sarkar},
   ISSN = {2047-4938},
   language = {English},
   title = {Employing Fusion of Learned and Handcrafted Features for Unconstrained Ear  Recognition},
   journal = {IET Biometrics},
   year = {2018},
   month = {January},
   publisher ={Institution of Engineering and Technology},
   copyright = {© Institution of Engineering and Technology},
   url = {http://digital-library.theiet.org/content/journals/10.1049/iet-bmt.2017.0210}
}