L02-Data Warehouse Components

Data Warehouse Components Data Warehouse and Business Intelligence
Data Warehouse
Components
Samuel I. G. Situmeang
Modified slides provided by: Michael A. Fudge, Jr.

Data Warehousing, Syracuse University, 2017
Lecture Objectives
• CIF Components
2
Recall: Inmon’s CIF
The CIF is a
reference architecture
3
Understanding the Diagram
Data
Stores Applications
Components
Processes
The CIF is a
4
CIF Components
5
External World & Applications
The CIF is a
6
External World & Applications
• External World – the people and systems that generate operational

data.
• Applications – the systems which provide the source for the
operational data.
• Examples: ERP’s, Business Applications, Internet data, external data
streams.
• These are the inputs and data sources for the CIF.
• OLTP Systems – Operational data, transaction-oriented.
7
Integration & Transformation Layer
The CIF is a
8
Integration & Transformation Layer
• I&T layer – takes un-integrated data from multiple sources and

integrates and consolidates it.
• Computer programs are written to transform data from the external
world into corporate data.
• The data come from a variety of sources and in both structured and
un-structured formats.
• Today’s Database Management Systems provide tooling to assist with
this process.
• This is the most difficult and time-consuming component of the CIF.
• Two approaches: ETL and ELT.
9
ETL – Extract Transform Load
• The data transformation occurs over staged data.

• The source data is not stored in the warehouse.
10
ELT – Extract Load Transform
• The data transformation occurs over warehoused data.

• The staged data is stored in the warehouse.
11
Operational Data Store
The CIF is a
12
Operational Data Store
• Integrated, detailed, and current data from the External World and
Applications.
• Consolidated from disparate sources.
• Does not grow over time.
• Performs similarly to a transactional database.
• Structured differently than a data warehouse, and therefore should be
stored as a separate database.
• Receives data from I&T layer sends data to the data warehouse.
• The data warehouse can populate it, too.
• Think of it as a consolidated operational database.
13
Enterprise Data Warehouse
The CIF is a
14
Enterprise Data Warehouse
• Subject-oriented, integrated, summarized, and current data from

the External World and Applications.
• Optimized for query performance.
• Structured differently than operational data, typically in a
dimensional model.
• Receives data from I&T layer and the ODS.
• Use as a source for data marts and decision support systems.
• Grows in size over time due to historical data.
• The heart of the CIF.
15
ODS vs. EDW

No Characteristic Operational Data Store Data Warehouse
1 Primary Purpose Run the business on a current basis Support managerial decision making
2 Design Goal Performance throughput, availability Easy reporting and analytics
3 Primary Users Clerks, salespersons, administrators Managers, business analysis,

customers
4 Subject-Oriented Yes Yes
5 Integrated Yes Yes
6 Detailed Data Yes Yes
7 Summary Data No Yes
8 Time of Data Current data Historical snapshots
9 Updates Frequent small updates Periodic batch updates
10 Queries Simple queries on a few rows Complex queries on several rows
16
Why No ODS in the EDW?
I need fast
updates!
I need query
performance!
You can’t have both! (Think of the Index!)
17
Data Marts
The CIF is a
18
Data Marts
• A collection of data tailored to the informational needs of a

department or business process.
• Easy to control, low cost, and customizable due to their limited
scope.
• Receive their inputs from the Enterprise Data Warehouse.
• Are source data for Online Analytical Processing (OLAP) engines.
19
OLAP
ROLAP MOLAP
• Uses a Relational Database • Uses a Multi-Dimensional
Management System Database Management System
• Data design is the Star Schema • Data design is the Cube
• Built on well-known relational • Highly flexible, includes
concepts Metadata.
• In the EDW. • Data Marts
Typical implementations have the

ROLAP star schema feed the MOLAP cube
20
ROLAP – Star Schema
• Stored in a
relational DBMS
• Fact table is M-M
relationship among
dimensions.
21
MOLAP - Cube
• Stored in a Multi-
Dimensional DBMS
• Facts are pre-
aggregated across all
dimensions for
improved
performance.
• Metadata: Drill down
hierarchy and
Identified Facts
22
DSS Applications
The CIF is a
23
Decision-Support Systems
• Business Intelligence.
• Front-ends to ROLAP and MOLAP Engines.
• Help us explore and visualize information at a high level.
24
Cross-Media Storage
The CIF is a
25
Cross-Media Storage Manager
• Stores historical data which is infrequently accessed.

• Moved out of the EDW, which has high-end, performant storage into
more affordable storage with less performant access times.
• A process exists to enable some transparency in the retrieval process.
26
In Summary…
• The CIF is a reference architecture for building out an information

ecosystem.
• Applications from the external world are inputs into the CIF.
• The Integration & Transformation Layer transforms transactional
data into corporate data.
• The Operational Data Store contains consolidated, non-historical
data.
• The Enterprise Data Warehouse contains consolidated historical
data.
• Data marts are tailored to the informational needs of a department
or business process.
27
EOF
28

L02-Data Warehouse Components

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

L02-Data Warehouse Components

Uploaded by

Copyright:

Available Formats

Data Warehouse Components Data Warehouse and Business Intelligence

Modified slides provided by: Michael A. Fudge, Jr.

Recall: Inmon’s CIF

Understanding the Diagram

External World & Applications

External World & Applications

• External World – the people and systems that generate operational

Integration & Transformation Layer

Integration & Transformation Layer

• I&T layer – takes un-integrated data from multiple sources and

ETL – Extract Transform Load

• The data transformation occurs over staged data.

ELT – Extract Load Transform

• The data transformation occurs over warehoused data.

Operational Data Store

Operational Data Store

Enterprise Data Warehouse

Enterprise Data Warehouse

• Subject-oriented, integrated, summarized, and current data from

ODS vs. EDW

2 Design Goal Performance throughput, availability Easy reporting and analytics

3 Primary Users Clerks, salespersons, administrators Managers, business analysis,

5 Integrated Yes Yes

6 Detailed Data Yes Yes

7 Summary Data No Yes

8 Time of Data Current data Historical snapshots

9 Updates Frequent small updates Periodic batch updates

10 Queries Simple queries on a few rows Complex queries on several rows

Why No ODS in the EDW?

You can’t have both! (Think of the Index!)

• A collection of data tailored to the informational needs of a

Typical implementations have the

ROLAP – Star Schema

Cross-Media Storage Manager

• Stores historical data which is infrequently accessed.

• The CIF is a reference architecture for building out an information

You might also like