Data Transformation

Data      Transformation:

Transforming the data from one format to client required format/business standard format

-A staging is a temporary memory or buffer where the following data transformation activities takes place

•        Data merging

•        Data Cleansing

•        Data scrubbing

•        Data aggregation

 

 

Data Merging: It’s a process of integrating the data from multiple

data sources into a single operational data set

•         Sources can be homogenous/heterogenous

Two types of merging:

a)Horizontal Merging: Process of joining the records horizontally

when the 2 sources are having different data definitions

Ex: Source1---Emp Table

Empno-Ename-Sal-Deptno

Source2----Dept Table

Deptno-Dname-Loc

b)Vertical Merging: Process of joining the records vertically when the 2

sources are having similar data structures

 

Data cleansing: is a process of converting inconsistencies into

consistencies and removing unwanted data

Data scrubbing: is a process of delivering new data

definitions according to the user requirements

Data aggregation: is a process of calculating the

summaries from the detailed data

•         Data scrubbing is applied for each record

•         Data aggregation is applied for a group of records

Data Loading: Dumping/inserting of the processed data into the target system(DWH)

2 types of data loads:

a)Initial/Full load: is a process of loading the data into an empty

target table

b)Incremental/Delta load: is a process of loading only new

records/changed records, which takes place after initial load

•         First we load dimensions

•         Next we load facts