<script type="application/ld+json">
{
 "@context": "https://schema.org",
 "@type": "FAQPage",
 "mainEntity": {
   "@type": "Question",
   "name": "What is a Recurrent Neural Network?",
   "acceptedAnswer": {
     "@type": "Answer",
     "text": "Recurrent Neural Network(RNN) is a type of Neural Network where the output from the previous step is fed as input to the current step. In traditional neural networks, all the inputs and outputs are independent of each other."
   }
 }
}
</script>

Recurrent neural network

What is a Recurrent Neural Network?

Recurrent Neural Network(RNN) is a type of Neural Network where the output from previous step are fed as input to the current step. In traditional neural networks, all the inputs and outputs are independent of each other. 

In cases like when it is required to predict the next word of a sentence, the previous words are required and hence there is a need to remember the previous words. Thus RNN came into existence, which solved this issue with the help of a Hidden Layer. The main and most important feature of RNN is Hidden state, which remembers some information about a sequence.

RNN have a “memory” which remembers all information about what has been calculated. It uses the same parameters for each input as it performs the same task on all the inputs or hidden layers to produce the output. This reduces the complexity of parameters, unlike other neural networks.

How do Recurrent Neural Networks work?

Training a typical neural network involves the following steps:

  1. Input an example from a dataset.
  2. The network will take that example and apply some complex computations to it using randomly initialised variables (called weights and biases).
  3. A predicted result will be produced.
  4. Comparing that result to the expected value will give us an error.
  5. Propagating the error back through the same path will adjust the variables.
  6. Steps 1–5 are repeated until we are confident to say that our variables are well-defined.
  7. A predication is made by applying these variables to a new unseen input.

Recurrent neural networks work similarly but, in order to get a clear understanding of the difference, we will go through the simplest model using the task of predicting the next word in a sequence based on the previous ones.

First, we need to train the network using a large dataset. For the purpose, we can choose any large text (“War and Peace” by Leo Tolstoy is a good choice). When done training, we can input the sentence “Napoleon was the Emperor of…” and expect a reasonable prediction based on the knowledge from the book.

So, how do we start? As explained above, we input one example at a time and produce one result, both of which are single words. The difference with a feedforward network comes in the fact that we also need to be informed about the previous inputs before evaluating the result. So you can view RNNs as multiple feedforward neural networks, passing information from one to the other.

Where can we use Recurrent Neural Networks?

RNNs are powerful machine learning models and have found use in a wide range of areas. It is distinctly different from CNN models like GoogleNet. In this article, we have explored the different applications of RNNs in detail. The main focus of RNNs is to use sequential data.

RNNs are widely used in the following domains/ applications:

  • Prediction problems
  • Language Modelling and Generating Text
  • Machine Translation
  • Speech Recognition
  • Generating Image Descriptions
  • Video Tagging
  • Text Summarization
  • Call Center Analysis
  • Face detection, OCR Applications as Image Recognition
  • Other applications like Music composition
About Engati

Engati powers 45,000+ chatbot & live chat solutions in 50+ languages across the world.

We aim to empower you to create the best customer experiences you could imagine. 

So, are you ready to create unbelievably smooth experiences?

Check us out!

Recurrent neural network

October 14, 2020

Table of contents

Key takeawaysCollaboration platforms are essential to the new way of workingEmployees prefer engati over emailEmployees play a growing part in software purchasing decisionsThe future of work is collaborativeMethodology

What is a Recurrent Neural Network?

Recurrent Neural Network(RNN) is a type of Neural Network where the output from previous step are fed as input to the current step. In traditional neural networks, all the inputs and outputs are independent of each other. 

In cases like when it is required to predict the next word of a sentence, the previous words are required and hence there is a need to remember the previous words. Thus RNN came into existence, which solved this issue with the help of a Hidden Layer. The main and most important feature of RNN is Hidden state, which remembers some information about a sequence.

RNN have a “memory” which remembers all information about what has been calculated. It uses the same parameters for each input as it performs the same task on all the inputs or hidden layers to produce the output. This reduces the complexity of parameters, unlike other neural networks.

How do Recurrent Neural Networks work?

Training a typical neural network involves the following steps:

  1. Input an example from a dataset.
  2. The network will take that example and apply some complex computations to it using randomly initialised variables (called weights and biases).
  3. A predicted result will be produced.
  4. Comparing that result to the expected value will give us an error.
  5. Propagating the error back through the same path will adjust the variables.
  6. Steps 1–5 are repeated until we are confident to say that our variables are well-defined.
  7. A predication is made by applying these variables to a new unseen input.

Recurrent neural networks work similarly but, in order to get a clear understanding of the difference, we will go through the simplest model using the task of predicting the next word in a sequence based on the previous ones.

First, we need to train the network using a large dataset. For the purpose, we can choose any large text (“War and Peace” by Leo Tolstoy is a good choice). When done training, we can input the sentence “Napoleon was the Emperor of…” and expect a reasonable prediction based on the knowledge from the book.

So, how do we start? As explained above, we input one example at a time and produce one result, both of which are single words. The difference with a feedforward network comes in the fact that we also need to be informed about the previous inputs before evaluating the result. So you can view RNNs as multiple feedforward neural networks, passing information from one to the other.

Where can we use Recurrent Neural Networks?

RNNs are powerful machine learning models and have found use in a wide range of areas. It is distinctly different from CNN models like GoogleNet. In this article, we have explored the different applications of RNNs in detail. The main focus of RNNs is to use sequential data.

RNNs are widely used in the following domains/ applications:

  • Prediction problems
  • Language Modelling and Generating Text
  • Machine Translation
  • Speech Recognition
  • Generating Image Descriptions
  • Video Tagging
  • Text Summarization
  • Call Center Analysis
  • Face detection, OCR Applications as Image Recognition
  • Other applications like Music composition
Share

Continue Reading