<script type="application/ld+json">
{
 "@context": "https://schema.org",
 "@type": "FAQPage",
 "mainEntity": [{
   "@type": "Question",
   "name": "What is knowledge extraction?",
   "acceptedAnswer": {
     "@type": "Answer",
     "text": "he use of a linguistic representation for expressing knowledge acquired by learning systems is an important issue as regards to user understanding. Under this assumption, and to make sure that these systems will be welcome and used, several techniques have been developed by the artificial intelligence community, under both the symbolic and the connectionist approaches."
   }
 },{
   "@type": "Question",
   "name": "What are Knowledge extraction techniques?",
   "acceptedAnswer": {
     "@type": "Answer",
     "text": "1. Knowledge graph completion: link prediction.
2. Triplet extraction from raw text.
3. Schema-based supervised learning."
   }
 }]
}
</script>

Knowledge extraction

What is knowledge extraction?

The use of a linguistic representation for expressing knowledge acquired by learning systems is an important issue as regards to user understanding. Under this assumption, and to make sure that these systems will be welcome and used, several techniques have been developed by the artificial intelligence community, under both the symbolic and the connectionist approaches. 

Knowledge extraction is the creation of knowledge from structured (relational databases, XML) and unstructured (text, documents, images) sources. The resulting knowledge needs to be in a machine-readable and machine-interpretable format and must represent knowledge in a manner that facilitates inferencing. Although it is methodically similar to information extraction (NLP) and ETL (data warehouse), the main criteria is that the extraction result goes beyond the creation of structured information or the transformation into a relational schema. It requires either the reuse of existing formal knowledge (reusing identifiers or ontologies) or the generation of a schema based on the source data.

Knowledge extraction techniques

1. Knowledge graph completion: link prediction

Translating Embeddings for Modelling Multi-relational Data by Bordes et al. in 2013 is a first attempt of a dedicated method for KG completion. It learns an embedding for the entities and the relations in the same low-dimensional vector space. The objective function is such that it constraints entity e2 to be close to e1 + r. This is done by assigning a higher score to exist triplets than to random triplets obtained with negative sampling. This model is known as TransE and this work is related to the word2vecwork by Mikolov where relations between concepts naturally take the form of translations in the embedding space as seen in the picture here.

2. Triplet extraction from raw text

Triplet extraction can be done in a purely unsupervised way. Usually, the text is first parsed with several tools (such as TreeBank parser, MiniPar or OpenNLP parser) then the texts between entities (as well as the annotations from the parsers) are clustered and finally simplified. While attractive at the first look as no supervision is needed, there are a few drawbacks. 

First, it requires lots of tedious work to hand-craft rules which depend on the parser used. Moreover, the clusters found contain semantically related relations but they do not give us fine-grained implications. Typically, a cluster may contain “ is-capital-of “ and “ is-city-of “ which are semantically closed relations. However, with the unsupervised approach, we will fail to discover that “ is-capital-of “ implies the relation “ is-city-of “ and not the opposite.

3. Schema-based supervised learning

In this case, the available data is a collection of sentences where each sentence is annotated with the triplet extracted from it. This means that raw text aligned with a KG of the text. Two recent papers (both published in 2016) give cutting-edge solutions to this problem.

The End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures article by Miwa and Bansal shows an approach that uses two stacked networks: a Bidirectional LSTM for entity detection (it creates an embedding of the entities) and a Tree-based LSTM for the detection of the relation that links the entities found. The figure below from the original paper shows the architecture used.

 

Thanks for reading! We hope you found this helpful.


Ready to level-up your business? Click here.

About Engati

Engati powers 45,000+ chatbot & live chat solutions in 50+ languages across the world.

We aim to empower you to create the best customer experiences you could imagine. 

So, are you ready to create unbelievably smooth experiences?

Check us out!

Knowledge extraction

October 14, 2020

Table of contents

Key takeawaysCollaboration platforms are essential to the new way of workingEmployees prefer engati over emailEmployees play a growing part in software purchasing decisionsThe future of work is collaborativeMethodology

What is knowledge extraction?

The use of a linguistic representation for expressing knowledge acquired by learning systems is an important issue as regards to user understanding. Under this assumption, and to make sure that these systems will be welcome and used, several techniques have been developed by the artificial intelligence community, under both the symbolic and the connectionist approaches. 

Knowledge extraction is the creation of knowledge from structured (relational databases, XML) and unstructured (text, documents, images) sources. The resulting knowledge needs to be in a machine-readable and machine-interpretable format and must represent knowledge in a manner that facilitates inferencing. Although it is methodically similar to information extraction (NLP) and ETL (data warehouse), the main criteria is that the extraction result goes beyond the creation of structured information or the transformation into a relational schema. It requires either the reuse of existing formal knowledge (reusing identifiers or ontologies) or the generation of a schema based on the source data.

Knowledge extraction techniques

1. Knowledge graph completion: link prediction

Translating Embeddings for Modelling Multi-relational Data by Bordes et al. in 2013 is a first attempt of a dedicated method for KG completion. It learns an embedding for the entities and the relations in the same low-dimensional vector space. The objective function is such that it constraints entity e2 to be close to e1 + r. This is done by assigning a higher score to exist triplets than to random triplets obtained with negative sampling. This model is known as TransE and this work is related to the word2vecwork by Mikolov where relations between concepts naturally take the form of translations in the embedding space as seen in the picture here.

2. Triplet extraction from raw text

Triplet extraction can be done in a purely unsupervised way. Usually, the text is first parsed with several tools (such as TreeBank parser, MiniPar or OpenNLP parser) then the texts between entities (as well as the annotations from the parsers) are clustered and finally simplified. While attractive at the first look as no supervision is needed, there are a few drawbacks. 

First, it requires lots of tedious work to hand-craft rules which depend on the parser used. Moreover, the clusters found contain semantically related relations but they do not give us fine-grained implications. Typically, a cluster may contain “ is-capital-of “ and “ is-city-of “ which are semantically closed relations. However, with the unsupervised approach, we will fail to discover that “ is-capital-of “ implies the relation “ is-city-of “ and not the opposite.

3. Schema-based supervised learning

In this case, the available data is a collection of sentences where each sentence is annotated with the triplet extracted from it. This means that raw text aligned with a KG of the text. Two recent papers (both published in 2016) give cutting-edge solutions to this problem.

The End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures article by Miwa and Bansal shows an approach that uses two stacked networks: a Bidirectional LSTM for entity detection (it creates an embedding of the entities) and a Tree-based LSTM for the detection of the relation that links the entities found. The figure below from the original paper shows the architecture used.

 

Thanks for reading! We hope you found this helpful.


Ready to level-up your business? Click here.

Share

Continue Reading