Exploring GPT-3: A quick tour of this powerful language model
On the 11th of June, 2020, the world was not ready for OpenAI to release a beta version of their latest language training model, GPT-3. Here’s a quick tour of this powerful language model.
What is GPT-3?
GPT-3, or Generative Pre-trained Transformer 3 is the latest breakthrough in language generators. It uses deep learning technologies to generate human-like texts. It’s the largest language model to exist so far, with over 175 billion parameters. In comparison, its predecessor, GPT-2 has only 1.5 billion parameters.
Here’s a graph comparing the number of parameters of other pre-trained NLP (Natural Language Processing) models.
How does GPT-3 work?
With over 175 billion parameters, it’s no surprise that GPT-3 is one of the most sophisticated text predictors to date.
But how is this possible?
Let’s go back to basics for a second. Remember when we first got introduced to transformer models back in 2017? They essentially used a deep learning technique called neural networks, to handle natural language data for translations and text summarisations. This transformer became the standard for all language generators, so GPT-3 followed suit. GPT-3 models are based on the same deep-learning transformer-based neural network architecture.
It’s the same standard, but it’s larger. And is pre-trained by an abundance of resources via datasets, such as Common Crawl, Wikipedia, WebText2, Books1, and Books2. The amount of training removes the need for human intervention and saves a lot of time. It’s like having the entire internet’s history in your generator.
Every article, every resource, every piece of information, all in on one powerful machine.
And with abundance of information, GPT-3 can generate the most statistically likely response for any given input, based on what it has learnt from the internet. Rich insights can be extracted from patterns in large data sets; And it can identify and apply linguistic patterns at a speed beyond what any human can do. Think of GPT-3 as a sort of virtual assistant.
What can GPT-3 accomplish?
The short answer is, a lot. But here’s a narrow list of what we believe GPT-3 can do.
- It could possibly write creative fiction
Researchers found that GPT-3's creative work samples were creative, thought-provoking, and were stylish, in a literary sense.
- It could make chatbot technology more sophisticated
What makes GPT-3 stand out is that it requires minimal context and input. After feeding the generator input, GPT-3 best guesses what the next chunk of text should be. It then repeats this process- taking the original input, plus the newly generated chunk and subsequently generating another chunk of text. It makes it the perfect virtual assistant for customers and employees.
GPT-3 is currently in beta, but we at Engati are excited for how things will evolve in the future. In the next series of blogs, we’re going to talk about the various use cases for GPT-3 in chatbots.
The future of chatbot technology looks bright with GPT-3 at our side, so stay tuned for the next one!
GPT-3's full version has a capacity of 175 billion machine learning parameters.
Until then, explore our current chatbot offerings with Engati.
Register to stay ahead of the conversational automation curve.
Engati is a one-stop platform for delighted customers. With our intelligent bots, we help you create the smoothest of Customer Experiences. And now, we're even helping you answer your customers' most complicated questions in real-time with Engati Live Chat. So, let's get started?Get Started Free