Tech Corner

Applications of Generative Language Models

Anwesh Roy
.
Jul 15
.
2-3 mins

Table of contents

Key takeawaysCollaboration platforms are essential to the new way of workingEmployees prefer engati over emailEmployees play a growing part in software purchasing decisionsThe future of work is collaborativeMethodology

Large language models like GPT-3 have stormed the NLP area in the last few years and have given us hope that we can solve some of the toughest problems.

The race to build the best language model has resulted in a so-called ‘arms race’ where bigger seems to be better so far. 

Modern day applications of Generative Language Models

1

GPT-3 and its impact on other models

GPT-3 by Open AI is the most well-known and most anticipated among other popular language models. It has been trained with 175 billion parameters. Many applications have been built with GPT-3 to demonstrate its capabilities, more for fun than profit.

But it does show the path forward in NLP and democratizes applications that build with NLP at the core. For NLP to come to the forefront in everyday applications, zero-shot learning, and few-shot learning is the way forward.

The recent news about Copilot, a code completion and code recommendation tool from Github, has spotlighted generative language models and their capabilities. Copilot could be the first commercial use of such models in a big way.

2

Microsoft’s Turing-NLG

Others are not staying behind. Microsoft has built Turing-NLG, a 17 billion parameter model already used in Bing’s search engine to improve auto-suggestion and phrase prediction. This is a well-known commercial application of generative language models.

Even though this model is 10X downscaled compared to GPT-3, the fact that Microsoft has found it to be useful enough to use in a search engine demonstrates the capabilities for such models.

3

Google’s Meena

Google had made an announcement of 1.6 trillion parameter language model early this year. This has been built by the Google Brain team using an efficient transformer based algorithm called Switch Transformers. 

This gives the hope that the cost to train and use such huge language models can be brought down making them economically feasible for small and medium companies working in the NLP area. This should give a huge boost to building more applications based on generative models.

Although application of this 1.6T language model has not been published as yet, Google has claimed in the past of using BERT based transformer models in Google search engine. 

So, it is very likely that this will find its way into Google’s search engine in the near future.

Google’s Meena chabot model is yet another application of a large language model which has been trained in conversation data. This is a 2.6 billion parameter model and was trained in multi-turn conversations which is one of the most difficult tasks for a chatbot.

The latest development in this area is the news about a 1.75 trillion parameters language model, called WuDao 2.0, built by Beijing Academy of Artificial Intelligence that understands English and Chinese and is claimed to simulate conversations, understand pictures, write poems and even create recipes.

The future of Generative Language Models

Currently, Open AI is pursuing commercialization of GPT-3. One good example of this is Viable which has built a product to find answers to user queries buried deep in customer surveys, product reviews and help desk data.

Similarly, Engati uses DocuSense Technology to provide easy access to relevant information. The engine parses through the given knowledge base and fetches the required information requested by the user. Engati strongly believes in ease, which is why this is carried out with a no code approach.

Try DocuSense today!

Share
Share

Anwesh Roy

Anwesh is the Senior Vice President of Engati. Driven by a passion to deliver value through AI-driven solutions, Anwesh is on a mission to mainstream Natural Language Processing (NLP), Natural Language Understanding (NLU), Natural Language Generation (NLG) and Data Analytics applications.

Andy is the Co-Founder and CIO of SwissCognitive - The Global AI Hub. He’s also the President of the Swiss IT Leadership Forum.

Andy is a digital enterprise leader and is transforming business strategies keeping the best interests of shareholders, customers, and employees in mind.

Follow him for your daily dose of AI news and thoughts on using AI to improve your business.

Catch our interview with Andy on AI in daily life

Continue Reading