Tech Corner

How will facial recognition systems & algorithms work in 2023?

Aniket Maurya
Sep 1
4-5 mins

Table of contents

Automate your business at $5/day with Engati

Switch to Engati: Smarter choice for WhatsApp Campaigns 🚀
facial recognition system

Building a robust facial recognition system that's free of bias both racial and gender is not an easy task. After all, algorithms don't create bias. We do.

The facial recognition technology market is growing rapidly. From airports in the United States relying on biometric data to screen international passengers, law enforcement depending on it to catch criminals, and social media using it to authenticate the user, facial recognition technology is the need of the hour.

The last decade was full of new state-of-the-art algorithms developed by top software development companies and groundbreaking research in the field of Deep Learning. New Computer vision algorithms were introduced as well. It all started when AlexNet, a Deep Convolutional Neural Network achieved high accuracy on the ImageNet dataset (dataset with more than 14 million images) in 2012. So what is facial recognition software? How does it work?

Before we dive into understanding how face recognition technology works, we need to understand how we recognize faces.

What is a Facial Recognition System?

Facial recognition is a technology-based method of recognizing a human face. Biometrics is used in a facial recognition system to map facial features from a photograph or video. To find a match, it compares the information to a database of known faces. Facial recognition can aid in the verification of a person's identity, but it also raises privacy concerns.

In 2022, the facial recognition market is expected to reach $7.7 billion, up from $4 billion in 2017. This is because facial recognition has a wide range of commercial applications. It can be used for a variety of purposes, including surveillance and marketing.

How do humans recognize a face?

Recognition systems in our brains are complex. In fact, scientists are still trying to figure it out. What we can assume is that the neurons in our brain first identify the face in the scene (from the person's body to its background), we extract the facial features, and store it in our own kind of database. Using our memory as a database, we can then classify the person according to their features. We have been trained on an infinitely large dataset and infinitely extensive neural network.

Facial Recognition software in machines is implemented the same way. First, we apply a facial detection algorithm to detect faces in the scene, extract facial features from the detected faces, and use an algorithm to classify the person.

How does the workflow of a Facial Recognition System work?

workflow of facial recognition algorithms
Workflow of facial recognition software


Face detection

Face detection is a specialized version of Object Detection, where there is only one object to detect - Human Face.

Just like computational time and space trade-offs in Computer Science, there's a trade-off between inference speed and accuracy in Machine Learning algorithms as well. There are many object detection algorithms out there, and different algorithms have their speed and accuracy trade-offs.

We evaluated different state-of-the-art best algorithms for face detection:

  • OpenCV (Haar-Cascade)
  • YoloV3 and Yolo-Tiny
  • SSD
  • BlazeFace
  • ShuffleNet and Faceboxes

To build a robust face detection system, we need an accurate and fast and best algorithm for facial recognition to run on a GPU as well as a mobile device in real-time.


In real-time inference on streaming video, people can have different poses, occlusions, and lighting effects on their face. It is important to precisely detect faces in various lighting conditions as well as poses.

facial recognition poses and lighting conditions
Detecting faces in various poses and lighting conditions

OpenCV (Haar-Cascade)

We started with Haar-cascade implementation of OpenCV, which is an open-source image manipulation library in C.

Pros: Since this library is written in C language. It is very fast for inference in real-time systems.

Cons: The problem with this implementation was that it was unable to detect side faces and performed poorly in different poses and lighting conditions.


This algorithm is based on Deep Learning methods. It uses Deep Cascaded Convolutional Neural Networks for detecting faces.

Pros: It had better accuracy than the OpenCV Haar-Cascade method

Cons: Higher run time 


YOLO face detection (You look only once) is the state-of-the-art Deep Learning algorithm for object detection. It has many convolutional neural networks, forming a Deep CNN model. (Deep means the model architecture complexity is enormous).

The original Yolo model can detect 80 different object classes with high accuracy. We used this Yolo facial recognition model for detecting only one object - the face.

We trained this algorithm on WiderFace (image dataset containing 393,703 face labels) dataset.

There is also a miniature version of the Yolo algorithm for face detection available, Yolo-Tiny. Yolo-Tiny takes less computation time by compromising its accuracy. We trained a Yolo-Tiny model with the same dataset, but the boundary box results were not consistent.

Pros: Very accurate, without any flaw. Faster than MTCNN.

Cons: Since it has colossal Deep Neural Network layers, it needs more computational resources. Thus, it is slow to run on the CPU or mobile devices. On GPU, it takes more VRAM because of its large architecture. 


SSD (Single Shot Detector) is also a deep convolutional neural network model like YOLO.

Pros: Good accuracy. It can detect in various poses, illumination, and occlusions. Good inference speed.

Cons: Inferior to YOLO model. Though inference speed was good it was still not adequate to run on CPU, low-end GPU, or mobile devices.


Like its name, it is a blazingly fast face-detection algorithm released by Google. It accepts 128x128 dimension image input. Its inference time is in sub-milliseconds. This algorithm is optimized to be used in face recognition on mobile phones. The reasons it is so fast are: 

  • It is a specialized face detector model, unlike YOLO and SSD, which were originally created to detect a large number of classes. Thus BlazeFace has a smaller Deep Convolutional Neural Network architecture than YOLO and SSD.
  • It uses Depthwise Separable Convolution instead of standard Convolution layers, which leads to fewer computations.

Pros: Very Good inference speed and accurate face detection.

Cons: This model is optimized for detecting facial images from a mobile phone camera, and thus it expects that face should cover most of the area in the image. It doesn’t work well when the face size is small. So in the case of CCTV camera images, it doesn’t perform well.


The latest face recognition algorithm we used is Faceboxes. Like BlazeFace, it is a Deep Convolutional Neural network with small architecture and designed just for one class - Human Face. Its inference time is real-time fast on CPU. Its accuracy is comparable to Yolo for face detection. It can detect small and large faces in an image precisely.

Pros: Fast inference speed and good accuracy.

Cons: Evaluation is in progress.


Feature extraction

After detecting faces in an image, we crop the faces and feed them to a Feature Extraction Algorithm, which creates face embedding- a multi-dimensional (mostly 128 or 512 dimensional) vector representing features of the face.

We used the FaceNet algorithm to create face-embeddings.

The embedding vectors represent the facial features of a person’s face. So embedding vectors of two different images of the same person will be closer and that of a different person will be farther. The distance between two vectors is calculated using Euclidean Distance.


Face classification

After getting the face-embedding vectors, we trained a classification algorithm, K-nearest neighbor (KNN), to classify the person from his embedding vector.

Suppose in an organization there are 1000 employees. We create face-embeddings of all the employees and use the embedding vectors to train a classification algorithm that accepts face-embedding vectors as input and returns the person's name.

A user could apply a filter that modifies​ specific pixels in an image before putting it on the web. These changes are imperceptible to the human eye but are very confusing for facial recognition algorithms - ThalesGroup

What are the applications of the Facial Recognition System?

application of facial recognition algorithm


People entering and exiting airports can be tracked using facial recognition systems. The technology has been used by the Department of Homeland Security to identify people who have overstayed their visas or are under criminal investigation.

Mobile phone companies

Face recognition was first used by Apple to unlock its iPhone X, and the technology was carried over to the iPhone XS. Face ID verifies that you are who you say you are when you access your phone. According to Apple, the odds of a random face unlocking your phone are one in a million.

Colleges & universities

In fact, facial recognition software can play a role. Your professor might find out if you skip class. Don't even consider having your bright roommate take your exam.

Social media

When you upload a photo to Facebook, it uses an algorithm to detect faces. If you want to tag people in your photos, the social media company will ask you. It can link to their profiles and recognize faces with an accuracy of 98%.

Marketing and advertisement campaigns

When marketing a product or an idea, marketers frequently consider factors such as gender, age, and ethnicity. Even at a concert, facial recognition can be used to identify specific audiences.

New tech brings new opportunities

Advancements in facial recognition systems and computer vision have taken great leaps. But this is only the beginning of the technological revolution. Imagine how powerful the duo of face recognition algorithms and chatbot technology would be!

It's never too late to become a part of this movement.

Aniket Maurya

Close Icon
Request a Demo!
Get started on Engati with the help of a personalised demo.
This is some text inside of a div block.
This is some text inside of a div block.
This is some text inside of a div block.
This is some text inside of a div block.
*only for sharing demo link on WhatsApp
Thanks for the information.
We will be shortly getting in touch with you.
Oops! something went wrong!
For any query reach out to us on
Close Icon
Congratulations! Your demo is recorded.

Select an option on how Engati can help you.

I am looking for a conversational AI engagement solution for the web and other channels.

I would like for a conversational AI engagement solution for WhatsApp as the primary channel

I am an e-commerce store with Shopify. I am looking for a conversational AI engagement solution for my business

I am looking to partner with Engati to build conversational AI solutions for other businesses

Close Icon
You're a step away from building your Al chatbot

How many customers do you expect to engage in a month?

Less Than 2000


More than 5000

Close Icon
Thanks for the information.

We will be shortly getting in touch with you.

Close Icon

Contact Us

Please fill in your details and we will contact you shortly.

This is some text inside of a div block.
This is some text inside of a div block.
This is some text inside of a div block.
This is some text inside of a div block.
This is some text inside of a div block.
Thanks for the information.
We will be shortly getting in touch with you.
Oops! Looks like there is a problem.
Never mind, drop us a mail at