<!-- JSON-LD markup generated by Google Structured Data Markup Helper. --><script type="application/ld+json">{  "@context" : "http://schema.org",  "@type" : "Article",  "name" : "Facial recognition algorithms",  "author" : {    "@type" : "Person",    "name" : "Aniket Maurya"  },  "image" : "https://global-uploads.webflow.com/5ef788f07804fb7d78a4127a/5ef788f17804fb6c75a41abd_Facial%20Recognition%20Algorithms.png",  "articleSection" : "We evaluated different state-of-the-art object detection algorithms",  "articleBody" : "OpenCV (Haar-Cascade)</LI><LI>MTCNN</LI><LI>YoloV3 and Yolo-Tiny</LI><LI>SSD</LI><LI>BlazeFace</LI><LI>ShuffleNet and Faceboxes",  "url" : "https://www.engati.com/blog/facial-recognition-systems" : {    "@type" : "Organization",    "name" : "Engati"  }}</script>

Facial recognition algorithms

Aniket Maurya
min read
Facial recognition algorithms


Building a robust facial Recognition system that is free of bias (racial or gender), is not an easy task. Algorithms don't create bias. It comes from humans.

The last decade was full of new state-of-the-art algorithms and groundbreaking research in the field of Deep Learning. New Computer vision algorithms were introduced.

It all started when AlexNet, a Deep (Convolutional) Neural Network that achieved high accuracy on ImageNet dataset (dataset with more than 14 million images) in 2012.

How do humans recognise a face?

Probably, the neurons in their brain first identify the face in the scene (from person body and background), extract the facial features and by those features classify the person. We have been trained on an infinitely large dataset and infinitely big neural network.

Facial Recognition in machines is implemented the same way. First, we apply a facial detection algorithm to detect faces in the scene, then extract facial features from the detected faces and use an algorithm to classify the person.

The workflow of a facial Recognition system is as follows:

workflow of facial recognition algorithms engati

Face detection

Face detection is a specialised version of Object Detection, where there is only one object to detect - Human Face.

Just like computational time and space trade-off in Computer Science, Machine Learning algorithms also a trade-off between inference speed and accuracy. There are many object detection algorithms out there and different algorithms have their speed and accuracy trade-off.

We evaluated different state-of-the-art object detection algorithms:

  1. OpenCV (Haar-Cascade)
  2. MTCNN
  3. YoloV3 and Yolo-Tiny
  4. SSD
  5. BlazeFace
  6. ShuffleNet and Faceboxes

To build a robust face detection system we need an algorithm that is both accurate and fast to run on a GPU as well as a mobile device in real-time.


In real-time inference on streaming video people can have different poses, occlusions and lighting effects on their face. It is important to precisely detect faces in different lighting conditions as well as poses.

Facial recognition accuracy

OpenCV (Haar-Cascade)

We started with Haar-cascade implementation of OpenCV, which is an open-source image manipulation library in C.

Pros: Since this library is written in C language. It is very fast for inference in real-time systems.

Cons: The problem with this implementation was that it was unable to detect side faces and performing poorly in different poses and lighting conditions.


This algorithm is based on Deep Learning methods. It uses Deep Cascaded Convolutional Neural Networks for detecting faces.

Pros: It had better accuracy then OpenCV Haar-Cascade method

Cons: Higher run time 


YOLO (You look only once) is the state-of-the-art Deep Learning algorithm for object detection. It has many convolutional neural networks, forming a Deep CNN model. (Deep means the model architecture complexity is very large).

The original Yolo model can detect 80 different object classes with high accuracy. We used this model for detecting only one object - Face.

We trained this algorithm on WiderFace (image dataset containing 393,703 face labels) dataset.

There is also a small version of the Yolo algorithm available, Yolo-Tiny. Yolo-Tiny takes less computation time by compromising its accuracy. We trained a Yolo-Tiny model with the same dataset but the boundary box results were not consistent.

Pros: Very accurate, without any flaw. Faster than MTCNN.

Cons:.Since it has very large Deep Neural Network layers so it needs more computational resources. Thus, it is slow to run on the CPU or mobile devices. On GPU it takes more VRAM because of its large architecture. 


SSD (Single Shot Detector) is also a deep convolutional neural network model like YOLO.

Pros: Good accuracy. Can detect in various poses, illumination and occlusions. Good inference speed.

Cons: Inferior to YOLO model. Though, inference speed was good but still not adequate enough to run on CPU, low-end GPU or mobile devices.


Like its name, it is a blazingly fast face-detection algorithm released by Google. It accepts 128x128 dimension image input. Its inference time is in sub-milliseconds. It is optimized to be used in mobile phones. The reasons it is so fast are: 

  • It is a specialized face detector model, unlike YOLO and SSD which were originally created for detecting a large number of classes. Thus BlazeFace has a smaller Deep Convolutional Neural Network architecture than YOLO and SSD.
  • It uses Depthwise Separable Convolution instead of standard Convolution layers which leads to fewer computations.

Pros: Very Good inference speed and accurate face detection.

Cons: This model is optimised for detecting faces from mobile phone camera and thus it expects that face should cover most of the area in the image. It doesn’t work well when the face size is small. So in case of CCTV camera images it doesn’t perform well.


The latest face detection algorithm we used is Faceboxes. Like BlazeFace, it is a Deep Convolutional Neural network with small architecture and designed just for one class - Human Face. Its inference time is real-time fast on CPU. Its accuracy is comparable to Yolo for face detection. It can detect small and large faces in an image, precisely.

Pros: Fast inference speed and good accuracy.

Cons: Evaluation is in progress.

Feature extraction

After detecting faces in an image, we crop the faces and feed it to a Feature Extraction Algorithm which creates face embedding- a multi-dimensional (mostly 128 or 512 dimensional) vector representing features of the face.

We used FaceNet algorithm to create face-embeddings.

The embedding vectors represent the facial features of a person’s face. So embedding vectors of two different images of the same person will be closer and that of a different person will be farther. The distance between two vectors are calculated using Euclidean Distance.

Face classification

After getting the face-embedding vectors we trained a classification algorithm, K-nearest neighbour (KNN), to classify the person from his embedding vector.

Suppose in an organization there are 1000 employees. We create face-embeddings of all the employees and use the embedding vectors to train a classification algorithm that accepts face-embedding vectors as input and returns the name of the person.

A user could apply a filter that modifies​ specific pixels in an image before putting it on the web. These changes are imperceptible to the human eye but are very confusing for facial recognition algorithms - ThalesGroup

New tech brings new opportunities

Advancements in facial recognition have taken great leaps. But this is only the beginning of the technological revolution. Imagine how powerful the duo of facial recognition algorithms and chatbot technology.

It's never too late to become a part of this movement.

Register with Engati today.


  1. Study finds gender and skin-type bias in commercial artificial-intelligence systems: https://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-021
  2. Euclidean distance
  3. OpenCV
  4. YoloV3
  5. SSD
  6. MTCNN
  7. BlazeFace
  8. FaceBoxes
  9. FaceNet
No items found.
About Engati

Engati is a one-stop platform for delighted customers. With our intelligent bots, we help you create the smoothest of Customer Experiences. And now, we're even helping you answer your customers' most complicated questions in real-time with Engati Live Chat. So, let's get started?

Get Started Free
Limited Time Offer - WhatsApp automation chatbot now available at a reduced price - 180 USD for 10K messages, 250 USD for 30K messages, 320 USD for 100K messages, all inclusive