Tech Corner

GPT-3: A quick tour of this powerful language model

Aish Sankhé
.
Oct 19
.
3-4 mins

Table of contents

Automate your business at $5/day with Engati

REQUEST A DEMO
engati gpt 3

On the 11th of June, 2020, the world was not ready for OpenAI to release a beta version of their latest language training model, GPT-3. Here’s a quick tour of this powerful language model.

What is GPT-3?

GPT-3, or Generative Pre-trained Transformer 3 is the latest breakthrough in language generators. It uses deep learning technologies to generate human-like texts. It’s the largest language model to exist so far, with over 175 billion parameters. In comparison, its predecessor, GPT-2 has only 1.5 billion parameters.

Here’s a graph comparing the number of parameters of other pre-trained NLP (Natural Language Processing) models.

The number of parameters GPT-3 can handle compared to GPT-2, NVDIA's Megatron, Microsoft Turing NLGT
The number of parameters GPT-3 can handle compared to other popular language models

How does GPT-3 work?

With over 175 billion parameters, it’s no surprise that GPT-3 is one of the most sophisticated text predictors to date.  

But how is this possible?

Let’s go back to basics for a second. Remember when we first got introduced to transformer models back in 2017? They essentially used a deep learning technique called neural networks, to handle natural language data for translations and text summarisations. This transformer became the standard for all language generators, so GPT-3 followed suit. GPT-3 models are based on the same deep-learning transformer-based neural network architecture.  

It’s the same standard, but it’s larger. And is pre-trained by an abundance of resources via datasets, such as Common Crawl, Wikipedia, WebText2, Books1, and Books2. The amount of training removes the need for human intervention and saves a lot of time. It’s like having the entire internet’s history in your generator.

Every article, every resource, every piece of information, all in on one powerful machine.

And with abundance of information, GPT-3 can generate the most statistically likely response for any given input, based on what it has learnt from the internet. Rich insights can be extracted from patterns in large data sets; And it can identify and apply linguistic patterns at a speed beyond what any human can do. Think of GPT-3 as a sort of virtual assistant.  

What can GPT-3 do?  

The short answer is, a lot. But here’s a narrow list of what we believe GPT-3 can do.  

  • It could possibly write creative fiction Researchers found that GPT-3's creative work samples were creative, thought-provoking, and were stylish, in a literary sense.
  • It could make chatbot technology more sophisticatedWhat makes GPT-3 stand out is that it requires minimal context and input. After feeding the generator input, GPT-3 best guesses what the next chunk of text should be. It then repeats this process- taking the original input, plus the newly generated chunk and subsequently generating another chunk of text. It makes it the perfect virtual assistant for customers and employees.

GPT-3 is currently in beta, but we at Engati are excited for how things will evolve in the future. In the next series of blogs, we’re going to talk about the various use cases for GPT-3 in chatbots.  

The future of chatbot technology looks bright with GPT-3 at our side, so stay tuned for the next one!

GPT-3's full version has a capacity of 175 billion machine learning parameters.
- Wikipedia

Until then, explore our current chatbot offerings with Engati.  

Register to stay ahead of the conversational automation curve.

Aish Sankhé

Aish Sankhé is a content writer and co-host for Engati CX, specializing in topics like Customer eXperience, Digital Transformation, and Technology.

Close Icon
Request a Demo!
Get started on Engati with the help of a personalised demo.
Thanks for the information.
We will be shortly getting in touch with you.
Oops! something went wrong!
For any query reach out to us on contact@engati.com
Close Icon
Congratulations! Your demo is recorded.

Select an option on how Engati can help you.

I am looking for a conversational AI engagement solution for the web and other channels.

I would like for a conversational AI engagement solution for WhatsApp as the primary channel

I am an e-commerce store with Shopify. I am looking for a conversational AI engagement solution for my business

I am looking to partner with Engati to build conversational AI solutions for other businesses

continue
Finish
Close Icon
You're a step away from building your Al chatbot

How many customers do you expect to engage in a month?

Less Than 2000

2000-5000

More than 5000

Finish
Close Icon
Thanks for the information.

We will be shortly getting in touch with you.

Close Icon

Contact Us

Please fill in your details and we will contact you shortly.

Thanks for the information.
We will be shortly getting in touch with you.
Oops! Looks like there is a problem.
Never mind, drop us a mail at contact@engati.com