Occam’s razor

What is Occam’s razor?

Occam’s razor (which is also spelled ‘Ockham’s razor’) is also referred to as the Law of Economy or the Law of Parsimony. It is a problem-solving principle that has been attributed to the 14th-century logician, friar, and scholastic philosopher William of Ockham.

William of Ockham states that “pluralitas non est ponenda sine necessitate”, or “plurality should not be posited without necessity”.

Essentially, the Law of Economy says that “Entities are not to be multiplied beyond necessity.”

This principle of philosophical reasoning advocates simplicity. 

It suggests that among two rival theories, the one with the fewest assumptions should be selected.

This is because the simplest theory will be much easier to verify and the simpler the solution, the less difficulty will be involved in executing and implementing it.

It is used widely in science. Even Isaac Newton, in Principia Mathematica, said that "We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances."

Basically, account for the observed facts, but keep your theories as simple as possible.

‍

How is Occam's razor applied in machine learning?

Some ML experts believe that Occam’s razor is helpful in designing machine learning models. They say that it helps in deciding which algorithms to use in the machine learning project and also in figuring out how to train the model with that algorithm.

When multiple suitable algorithms have comparable trade-offs, they opt for the one that is the simplest to deploy and easiest to interpret. 

Simplifying models by using dimensionality reduction and feature selection are considered to be examples of Occam’s razor in action in the sphere of machine learning.

Occam’s razor can also be cited regarding the problem of overfitting which occurs when models are far too complex to fit the data that is being analyzed and processed. An extremely complex model might be more fragile.

Is Occam’s razor always true?

Simplicity helps you clearly see what is most important in a phenomenon. Among two competing hypotheses, the simpler one might not always be the better option. Sometimes, it could be far less accurate.

Occam’s razor can only really be used if both the theories available predict identical results, and one of the theories makes fewer assumptions than the other one (is simpler than the other available hypothesis).

Science doesn’t tend to be that simple. It tends to be complex and messy. Using Occam’s razor can be tempting, but it might not always be the best way for you to proceed.

‍

Where is Occam’s Razor used?

Here are some of the areas in which Occam’s Razor is utilized:

The development of scientific theories

Scientists often make use of Occam’s Razor, especially for theoretical matters.The simpler a hypothesis is, the easier it is for the hypothesis to be proven or falsified. A complex explanation for a phenomenon involves several factors which can be hard to test or cause issues with the repeatability of an experiment. Because of this, the simplest solution which is consistent with the existing data is usually preferred. But, it is not uncommon for new data to cause hypotheses to become more complex over time. Scientists prefer to pick the simplest solution as the current data permits, while staying open to the possibility of future research allowing for greater complexity.

They also prefer simpler hypotheses because it is easier for them to obtain funding for simpler hypotheses since they are easier to prove.

Even Einstein made a reference to Occam’s Razor when he developed his theory of special relativity. He formulated his own version saying, “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” Or, “Everything should be made as simple as possible, but not simpler.”

Newton too used Occam’s Razor when he was developing his theories. He said “We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances.” He aimed to make his theories, including the three laws of motion, as simple as possible, with only the necessary minimum of underlying assumptions.

Medicine

Doctors make use of a version of Occam’s Razor, which states that they should look for the fewest possible causes to explain their patient’s multiple symptoms, and give preference to the most likely causes.

Making minimal diagnoses lowers the risk of over-treating a patient, causing panic, or causing dangerous interactions between different treatments. This is especially important within the current medical model, where patients are likely to see numerous health specialists and communication between them can be quite poor.

Prison Abolition and Fair Punishment

Occam’s razor has played a role for a long time in attitudes towards the punishment of crimes. Here it refers to the idea that people should be given the least punishment necessary for their crimes. The purpose of it is to avoi the excessive penal practices which were rather common in the past.

Jeremy Bentham, the founder of utilitarianism pioneered the concept of penal parsimony. His view was that punishments should not cause more pain than they prevent. In that view, life imprisonment for murder could be seen as justified because it could prevent a lot of potential pain if the perpetrator commits another offence, but imprisoning an impoverished person for a long period because of them stealing food would cause a great deal of suffering without really preventing any suffering.

Bentham’s thoughts regarding the application of Occam’s razor to punishment led to the prison abolition movement and several modern ideas related to rehabilitation.