Federated learning

What is federated learning?

Federated learning (also known as collaborative learning) is a machine learning technique that trains an algorithm across multiple decentralized edge devices or servers holding local data samples, without exchanging them. This approach stands in contrast to traditional centralized machine learning techniques where all the local datasets are uploaded to one server, as well as to more classical decentralized approaches which often assume that local data samples are identically distributed.

Federated learning enables multiple actors to build a common, robust machine learning model without sharing data, thus allowing to address critical issues such as data privacy, data security, data access rights, and access to heterogeneous data. Its applications are spread over a number of industries including defense, telecommunications, IoT, and pharmaceutics.

Federated learning makes it possible for mobile phones to learn a shared prediction model in collaboration wiht each other, while  keeping all the training data on device, this eliminating the need to store data on the cloud in order to perform machine learning.

‍

How does federated learning work?

Let’s take an example. Say you got selected as a machine learning intern in a company, and your task is to create a robust machine learning application, that needs to train itself on user-sensitive data.

You’re allowed to extract user data, aggregate it from many users, and stack them up on a centralized cloud server, for your model to crunch it. But it might infringe on someone’s privacy. 

You discussed it in a meeting, and your boss is now worried about the next steps. In the meantime, you and your teammates started discussing the matter.

One of them yelled, “What if we don’t take user-sensitive data, but train our model locally, on each device?”

We will train our model on the devices themselves, and not on the centralized server, that exposes sensitive data!  The local data generated by the user history, on a particular device, will now be used as on-device data to train our model and make it smarter, much quicker.

The pipeline goes as follows: 

• So, our centralized machine learning application will have a local copy on all devices, where users can use them according to our needs.
• The model will now gradually learn and train itself on the information inputted by the user and become smarter, time to time.
• The devices are then allowed to transfer the training results, from the local copy of the machine learning app, back to the central server.
• This same process happens across several devices, that have a local copy of the application. The results will be aggregated together in the centralized server, this time without user data.
• The centralized cloud server now updates its central machine learning model from the aggregated training results, which is now far better than the previously deployed version.
• The development team now updates the model to a newer version, and users update the application with the smarter model, created from their own data!

‍

What is federated learning platform?

A federated learning platform is a solution designed for data science on distributed and therefore non-centralized data. Federated learning techniques allow different companies to use their data together to jointly train machine learning models without having them directly sharing their data or centralizing it.

As such, computation requests are sent to the different data providers, and machine learning models are trained locally on the data provider's data. Then, only the locally trained model parameters are shared between the parties - meaning the underlying training data stays under the control of the data providers. A federated learning platform facilitates setting up distributed model training workflows while providing many other features that can enhance privacy-preserving measures and make the process of collaborating more efficient.

‍

What are the benefits of federated learning?

Here are some primary benefits of federated machine learning:

• FL enables devices like mobile phones to collaboratively learn a shared prediction model while keeping the training data on the device instead of requiring the data to be uploaded and stored on a central server.
• Moves model training to the edge, namely devices such as smartphones, tablets, IoT, or even “organizations” like hospitals that are required to operate under strict privacy constraints. Having personal data remain local is a strong security benefit.
• Makes real-time prediction possible, since prediction happens on the device itself. FL reduces the time lag that occurs due to transmitting raw data back to a central server and then shipping the results back to the device.
• Since the models reside on the device, the prediction process works even when there is no internet connectivity.
• FL reduces the amount of hardware infrastructure required. FL uses minimal hardware and what is available in mobile devices is more than enough to run the FL models.

‍

Is federated learning better?

Accurate machine learning models are valuable to companies and traditional centralized machine learning approaches have shortcomings like lack of continual learning on edge devices and aggregating private data on central servers. These are alleviated by federated learning.

In traditional machine learning, a central ML model is built using all available training data in a centralized environment. This works without any issues when a central server can serve the predictions.

However, in mobile computing, users demand fast responses and the communication time between user device and a central server may be too slow for a good user experience. To overcome this, the model may be placed in the end user device but then continual learning becomes a challenge since models are trained on a complete data set and the end user device does not have access to the complete dataset.

Another challenge with traditional machine learning is that user’s data gets aggregated in a central location for machine learning training which may be against the privacy policies of certain countries and may make the data more vulnerable to data breaches

Federated learning overcomes these challenges by enabling continual learning on end-user devices while ensuring that end user data does not leave end-user devices.

What is the difference between distributed learning and federated learning?

Distributed machine learning algorithms are multi-nodal system that build training models by carrying out independent training on various nodes. Having a distributed training system speeds up the training process when the models need to be trained on an enormous amount of data.

When you’re working with big data, training time goes up exponentially, which makes scalability and online re-training an issue. That’s where distributed learning systems can be built and training can be parallelized to save time. 

Federated learning systems are fundamentally different from distributed learning. Their main purpose is to address data privacy. While in a traditional data science pipeline, the data gets collected to a single server and is utilized to build and train a centralized model, federated learning involves a centralized model using decentralized model training.

In federated learning systems, a seed parameter set is sent to independent nodes that contain data and the models are then trained on the local nodes making use of the data stored in these nodes. After the model is trained independently, every one of these updated model weights gets sent back to the central server in which they are combined to create a model with a far greater degree of efficiency. Making use of global training data improves the model efficiency substantially. This even makes sure that  the data in every node follows the data privacy policies and protects against any data leak or breach. 

Basically, the main difference between distributed learning and federated learning is that distributed learning involves making use of centralized data and distributing the model training to different nodes, while federated learning involves making use of decentralized data and training, while effectively having a central model.