Data fusion

What is ​​data fusion in machine learning?

Data fusion is the process of getting and integrating data from multiple sources to create information that is more useful, accurate, and consistent than data from any individual source could be.

It usually involves getting data from a range of sources of a single subject and then combining that data for the purpose of central analysis. 

Data fusion is used to get higher quality information, thus making it possible for developers to build increasingly sophisticated models, thus making it possible to learn more about a project.

‍

‍

What are the types of data fusion?

The types of data fusion are:

• Low data fusion
• Intermediate data fusion
• High data fusion
• Sensor fusion
  

Low, intermediate, and high data fusion are determined based on the processing stage at which fusion takes place. It can also be classified as low level, feature level, and decision level. Sensor infusion is a subset of information fusion and is also known as multi-sensor data fusion.

‍

What are the ​​levels for the Data Fusion Information Group Model?

The Joint Directors of Laboratories Data Fusion Group gives six levels of the ​​Data Fusion Information Group Model (DFIG Model). These are:

• Level 0: Source preprocessing aka Data Assessment 
• Level 1: Object assessment
• Level 2: Situation assessment 
• Level 3: Impact assessment aka Threat Refinement
• Level 4: Project refinement aka Resource Management
• Level 5: User refinement aka Cognitive Refinement
• Level 6: Mission Refinement aka Management
  

‍

What is the difference between data integration and data fusion?

In data integration, you would retreive and combine heterogeneous data as an incorporated form and structure. The process of data integration makes it possible for users, organizations, and applications to merge various types of data like datasets, tables, documents etc. Essentially, you could say that data integration refers to the combination of technical and business processes used to combine data from disparate sources into insightful and valuable information. It involves pulling trustworthy and meaningful data from a range of sources.

This data that is pulled from mutliple disparate sources is stored using a range of technolgies and gives you a unified view of the data. Data integration becomes rather important when you have to merge the systems of two companies or consolidate applications in one compay for the purpose of getting a unified view of getting the company’s data assets. The later initiative is generally called a data warehouse.

Data integration is all about meticulously and methodically combining data from several sources, to make it more useful and valuable than it was on its own. According to IBM, “Data integration is the combination of technical and business processes used to combine data from disparate sources into meaningful and valuable information.”

Data integrate includes data warehousing, data migration, enterprise application/information integration, master data management, etc.

This is different from data fusion because data fusion invovles pulling data from multiple sources to build increasingly sophisticated models and understand more about a project. It usually involves collecting data on a single subject from various sources and then combining all of it for central analysis. 

In data fusion, you would generally be fusing data at different abstraction levels and varying levels of uncertainty so that you can support a narrower set of application workloads.

Data fusion gets applied to a wide range of technologies. It’s widely used in motion, biometric, and environmental sensors. It’s used in inertial sensors such as accelerometers, gyroscopes, and magnetometers. You could even combine a tri-axial accelerometer, gyroscope, and magnetometer to make inertial measurement units for 9 degree of freedom tracking. This could be used for biomechanical modelling.

Sensor and data fusion gets used in everything from Earth resource monitoring  weather forecasting, vehicular traffic control, military target classification, and tracking.

The main difference between data fusion and data integration is that data fusion is about combining data residing in different sources to provide users with a unified view of them while data fusion is about collecting data from different sources but is not focusing on generating more consistent, accurate, and useful information than that provided by any individual data source. 

‍

What is cloud data fusion?

Cloud data fusion is a fully-managed data engineering product from Google Cloud. It makes it easy for customers to build and manage ETL/ELT data pipelines in a more efficient manner.

It tried to move the focus from code (in which data engineers end up spending an enormous amount of time building connectors from a source to a sink) to insights and action.

It is built on top of the open-source project and has a convenient  drag and drop user interface which makes it easier to build data pipelines.

It makes it easier to move data around. Since Cloud Data Fusion is built on top of the open-source CDAP project it comes along with upwards of 100 connectors and is still growing. Creating a data pipeline between a source and sink only really needs a few clicks.

It also focuses on making it possible to perform transformations without using any code. That’s why Cloud Data Fusion also has a set of built-in transformations that can be seamlessly applied to your data.

‍

What are the Features of Cloud Data Fusion?

Open-source

Since its built on top of CDAP, there’s a vast community of people building new connectors.

Accessible

You don’t need to have a background in coding in order to use Cloud Data Fusion.

Metadata

It allows you to look for  integrated datasets by technical and business metadata.

Flexible

If you can’t currently do something via the UI, you can add your own code to the product.