Professional Documents
Culture Documents
Master Data Managment MDM Reference Architecture Whitepaper
Master Data Managment MDM Reference Architecture Whitepaper
Reference
Architecture
Contents
What is MDM?
Use of MDM
MDM Hub Implementation Style
Transaction Style Implementation
MDM Reference Architecture
Key Components of the Reference Architecture
Large Scale and High Performance MDM based on Talend
Contact Details
01
What is MDM?
Entities like Customer, Product, Accounts, Suppliers, and Locations etc. are key
ingredients that fuel the enterprise engine. The processes used to operate the
enterprise depend on these entities and their quality (to what extent can these
entities be trusted). These key entities are referred to as ‘Master Data’.
Most enterprises are slowly realising the importance of improving the quality of
their master data and managing its lifecycle more efficiently to optimise business
and operations. The discipline/ capability (people, process and technology) of
managing the master data within an organisation is called Master Data
Management
02
Use of MDM
IBM in their book ‘Enterprise Master Data Management’ describes three patterns Figure 1:
how the managed master data entities can be used: Use of MDM
Colla borative
Collaborative Use: Us e
In this pattern of use, different business functions or teams of people collaborate Master Data
lifecycle
to manage the lifecycle of the master data. Defined workflows ensure adherence management
to processes and that the right approvals are sought. Role-based access control Use of
determines which roles can manage which aspect of the master data attributes.
MDM
Typically, in this pattern of use, the MDM system broadcasts the master data to
the transactional system(s). Managing catalogues of products is an example of this
Tra ns a ction
pattern of use.
Us e
Transaction Use:
Updates of
In this pattern of use the MDM system participates in end-to-end business Master
Ana lyti ca l Dat a Entit y
transactions. Multiple applications and users need quick access in order to read Us e
and update the master data entity; to initiate and complete business transactions
Initiation of
such as orders and payments etc. Here the MDM system exposes data services Busines s
that can participate within an orchestrated business process. MDM systems Transaction
supporting this method of use might have the need to support several hundred Management
transactions per second on millions of master data records. Information reports Cempletion of
Busines s
Transaction
Analytical Use:
Dat a for Analytical
Applications
In this pattern of use, the role of MDM is to produce Management Information
(MI) reports or to provide data to an analytical application such as a Data
Warehouse (DWH).
03
MDM Hub Implementation Style
When implementing an MDM system one of the key decisions to be made is the Transaction Style
choice of MDM Hub architecture. Gartner and others describe three styles of
MDM Hub and choice depends on multiple considerations, such as: MDM systems implemented using this style enable read/write operations on the
master data. In this architecture style the Hub stores and manages all attributes
Time and budget available
that define the master data entity. The Hub is regarded as the single, authoritative
Read or Read/Write capability on Master Data source of the master data. Changes to the master data are enriched, cleansed,
de-duplicated and pushed by the Hub to the various source systems.
Data consistency
Data fragmentation This architecture style is more expensive and time consuming compared to the
Registry style. In addition, it directly impacts source systems as these need to
change to either implement a two-way synchronisation to maintain consistency of
Registry Style master data, or devolve responsibility of the master data entities by sourcing the
data directly from the MDM Hub.
MDM systems implemented using this style provide a read-only view of the
master data for downstream systems. Here the MDM Hub stores only a thin slice
of the MDM Data (Source System Identifiers, Foreign Key Identifiers, key
attributes needed for matching). Other data attributes that make up the master
data entity are spread across multiple business systems. The Hub is responsible for
cleansing and de-duplicating the master data entity and maintaining a reference to
source master data attributes that it does not manage from the other business
systems.
The main advantage of this architecture style is the low time and cost required to
implement an MDM solution as well as the reduced impact on the source data
systems.
The key disadvantage is performance: the time required for the registry Hub to
aggregate data from multiple source systems and provide a complete master data
record.
04
Transaction Style Implementation
Transaction Style can be implemented as: Figure 2:
Transaction Style Implementation
Co-Existence
Master data entities co-exist in both the Hub and the source systems with a two
way synchronisation between the two. Master data entities can be created/
updated either in the MDM Hub or the source system. The MDM Hub
standardises, cleanses, enriches and de-duplicates the master data and pushes the
data back to the source systems to rationalise the master data they hold. Transaction
Style
Centralised Implementation
Here the Master data entity is stored and managed solely within the MDM Hub.
Creation and update of master data entity is only done via the MDM system.
Source systems interact with the MDM system when they require the master data Co-Ex is tence Hybrid Centra lis ed
record.
Hybrid
This is the ‘Goldilocks’ or hybrid architecture style. Here the Hub does not store all
attributes that define the master data as in Transaction Style but stores more than
the Registry Style. The main aim of the Hybrid is to resolve performance issues
faced by the Registry Style.
05
MDM Reference Architecture
The MDM Reference Architecture is a vendor-neutral, product-agnostic Refer-
ence Architecture that supports multiple methods of MDM use, and multiple
architecture styles. It aims to define the key architecture building blocks (ABB)
that will enable an organisation to govern and maintain accurate master data for
all users and applications.
The proposed Reference Architecture is guided and influenced by the following The Reference Architecture illustrated in Figure 3: MDM Reference Architecture
principles: provides architecture building blocks that can manage the lifecycle of master data
as well as the quality and integrity of the data, making master data actionable and
Securely managing complete lifecycle of the managed master data entity. providing data services to enable the consumption and distribution of data. The
The data must be managed in line with prevailing privacy and data protec- design is guided by the principles outlined above.
tion legislation.
Being the single authoritative source for the managed master data, assuring
the integrity, quality and uniqueness of the master data record.
06
MDM Reference Architecture
Figure 3:
MDM Reference Architecture
E xter n al Interna l
Sou rce s Applications
Master
Integration Data
Platform Respitory
Transport Routing
Survivorship
Choreography
& Data
Stewardshipp Data
Quality
MGT Master Dat a Inform ation
Man age m ent Integ rati on
Master Data
Event MGT
07
Key components of the Reference Architecture:
Transaction Systems and Data Sources Figure 4:
Information Integration - Data Integrity Services
These are internal and external transactional systems that interact with the master
data management system to request the master data entities or enrich the master
data entities. These systems may, in addition, store the master data information in a
Data Integrity Services
registry or Co-existence Style implementation.
Integration Platform
The integration platform enables the Transaction Systems and Data Sources to interact Data Analytics Cleansing Data Matching
with the Master Data Management system. Profiling Standardisation Services
08
Key components of the Reference Architecture:
Master Data Management Authoring:
Subcomponent to author, approve, manage, customise, and extend the definition
The Master Data Management component comprises of the following subcompo- of master data. This subcomponent supports the collaborative style of MDM use
nents and can be invoked as part of a workflow process that creates, updates and
approves master data.
Data Services:
This sub component provides a service interface to access the master data entities Data Quality Management:
managed by the MDM system. The service interface support techniques such as This subcomponent works with the data integrity service to validate, standardise
messaging, method calls, web services, and batch processing. and enforce data quality rules.
09
Large Scale and High Performance MDM based on Talend
How does one instantiate the logical Ref-
erence Architecture, composed of archi-
Enterprise Data Sources
tecture building blocks listed above, to
CRM Sales & Enrichment Data
provide a high performing and scaleable Ecommerce Data Source Lake
Master Data Management platform?
Figure 5:
Highly scaleable, Performant MDM Elastic Talend ESB Talend MDM Talend Data MGT Analytics
Search Platform
Lifecucle Data Rules
Key: Routing
MGT Profiling Repository
Data Data
Matching Data Data Warehouse
MDM
Mediation Integration Matching &
Enterprise Event MGT
(ETL) Survivorship
Application
Mongo
Mongo DB
Foundation
Studi Admin Monitoring Metadata Connectors
Console Console MGT
10
Large Scale and High Performance Customer MDM
Talend MDM Platform: Elastic
Elastic Search:
Talend
Search
Talend ESB: Elasticsearch enables high performance, distributed search supporting a two-step
Providing Integration and Data Services matching process. The first step of the matching process performs a
coarse-grained match followed by a fine-grained match using Talend Data Man-
Talend Data Management: agement's matching component.
Providing Data Integration and Integrity services
11
Contact Details
12
Thank you
Lancy Mendonca – Enterprise Architect
A Computer Science grad with a BSc in Physics, Lancy Mendonca has well over two
decades of experience in creating and contributing to IT architectures for the telecom
and healthcare industries.
After leading roles at Tech Mahindra, British Telecom and Carphone Warehouse, Lancy
was appointed to head up the Solution Architecture and Design team at TalkTalk.
There, he was on point in the delivery of a multi-million pound business
transformation project for TalkTalk’s fixed line business. After five years at TalkTalk,
Lancy took on the Business Support Systems brief at Tesco Telecoms. He later helped
to shape Truphone’s Enterprise Application strategy.
An avid reader, a dedicated supporter of church charities, and someone who donates
his time to supporting young learners, Lancy is passionate about many things, but
particularly the future of data and the possibilities it offers to modern businesses.