Transduction

What is transduction in machine learning?

Transduction or transductive inference in n logic, statistical inference, and machine learning is reasoning that is derived from observed,  specific (training) cases to specific (test) cases. 

The idea behind it is to avoid constructing a function to learn from a training set. In transductive learning, the training set is known in advance.

It was introduced by Vladimir Vapnik in the 1990s. He viewed transduction to be superior to induction because induction makes it necessary to solve a more general problem (inferring a function) before solving a more specific problem (computing outputs for new cases).

"When solving a problem of interest, do not solve a more general problem as an intermediate step. Try to get the answer that you really need but not a more general one."

-Vladimir Vapnik

Transduction is used in the domain of statistical learning to refer to predicting particular examples when specific examples from a domain are given.

Vladimir proposes a transductive model for sequence tagging and natural language modeling. Transductive learning is employed in NLP sequence prediction tasks, specifically in translation. The definitions appear to be more relaxed than the strict one-output-per-input of Goldberg and the FST.

Ed Grefenstette, et al. described transduction as mapping an input string to an output string.

Not all semi-supervised learning methods can be considered to be transductive in nature. This is because The inclusion of unlabelled samples in the training is not the primary characteristic of transductive learning, the avoidance of building a general model is the primary characteristic of transductive learning.

A wide range of Natural Language Processing (NLP) tasks can be considered to be transduction problems (learning how to convert one string into another). Machine translation can be looked at as a prototypical example of transduction.

String transduction is a central part of a range of NLP applications. Some of these applications include:

Transliteration

This involves producing words in a target form after taking examples in a source form.

Spelling Correction

This refers to producing the right word spelling when incorrect word spellings are given.

Inflectional Morphology

Inflectional morphology involves creating new sequences when source sequences and context is available.

Machine Translation

Machine translation is the process of crafting sequences of words in a target language after referring to examples in the source language.

Speech Recognition

This involves deriving sequences of text from sequences of audio that is available.

Text-to-Speech

This is also known as speech synthesis and involves creating audio sequences from text sequences that are available.

‍

What is the difference between inductive and transductive learning?

Transductive learning is contrasted with other types of learning like inductive learning and deductive learning. Here’s how it is different from inductive learning.

Induction refers to reasoning from observed training cases to general rules. These are then applied to the test cases. Transduction refers to reasoning from observed, specific cases to specific cases.

Inductive learning is no different from traditional supervised learning. It involves building and training an ML model based on a labelled training dataset that is available. This trained model is then used for the purpose of predicting the labels of a testing dataset that we have not encountered before.

The biggest difference between inductive and transductive learning is that in transductive learning, you encounter the training data as well as the testing dataset while training the model. In inductive learning, you only encounter the training data while training the model and then apply the learned model on a dataset which it has not seen before or encountered before.

Transduction does not involve building a predictive model. On the other hand, inductive learning involves building a predictive model.

Transduction can be an expensive learning process when new data points are added via an input stream. Because when new data points are added, you need to re-run the entire algorithm. However, Since inductive learning involves building a predictive model when new data points are added, you don’t need to rerun everything.

New data points can be labeled within a very short period with fewer computations in inductive learning. , 

‍

What are transduction algorithms?

Transduction algorithms can be classified into two categories: algorithms that aim to predict discreet labels to unlabeled points, and algorithms that seek to predict continuous labels for unlabeled points.

The algorithms that seek to assign discreet labels are usually derived by means of adding partial supervision to a clustering algorithm. The types of algorithms that can be used for this purpose are flat clustering and hierarchical clustering.

Hierarchical clustering can be further divided into two categories: algorithms that cluster by partitioning and algorithms that cluster by agglomerating. Algorithms that aim to regress continuous labels are usually derived through the process of adding partial supervision to a manifold learning algorithm.

Partitioning transduction

Partitioning transduction can be looked at as top-down transduction. It essentially is a semi-supervised extension of partition-based clustering. Any reasonable partitioning technique can be used with this algorithm.

Max flow min cut partitioning schemes are very widely used for this purpose.

Agglomerative transduction

Agglomerative transduction can be considered to be bottom-up transduction. This is essentially a semi-supervised extension of agglomerative clustering.

Manifold transduction

Mainfold transduction is still a very young field of research and is in its nascent stage.

‍

What are the disadvantages of transduction?

The most significant disadvantage of transduction is that you cannot make use of the information that you learn for the purpose of labeling new instances because you aren’t building a new model. So, whenever you have to classify a new set of instances, you’ll have to repeat the entire training. It makes a lot of sense only when your goals (the instances that you wish to understand) are specific.

Another disadvantage of transductive learning is that its effectiveness may reduce if you have a few very noisy samples in your data.

The integrity of the entire dataset, or at least most of it, is essentially a prerequisite for transductive learning.